
BUILDING KNOWLEDGE 
TO IMPROVE SOCIAL POLICY 


April 2015 


William Corrin 
Leigh M. Parise 
Oscar Cerna 
Zeest Haider 
Marie-Andree Somers 


CASE 

MANAGEMENT 
FOR STUDENTS 
AT RISK OF 
DROPPING OUT 

Implementation and 
Interim Impact Findings 
from the Communities In 
Schools Evaluation 



Case Management for Students at Risk of Dropping Out 

Implementation and Interim Impact Findings from the 
Communities In Schools Evaluation 


William Corrin 
Leigh M. Parise 
Oscar Cerna 
Zeest Haider 
Marie-Andree Somers 


April 2015 


mdrc 

BUILDING KNOWLEDGE 
TO IMPROVE SOCIAL POLICY 


This report is funded by the Edna McConnell Clark Foundation’s Social Innovation Fund (SIF), a 
key White House initiative and program of the Corporation for National and Community Service 
(CNCS). The Edna McConnell Clark Foundation’s Social Innovation Fund includes support from 
CNCS and 15 private co-investors: The Edna McConnell Clark Foundation, The Annie E. Casey 
Foundation, The Duke Endowment, The William and Flora Hewlett Foundation, The JPB 
Foundation, George Kaiser Family Foundation, The Kresge Foundation, Open Society Founda- 
tions, The Penzance Foundation, The Samberg Family Foundation, The Charles and Lynn 
Schusterman Family Foundation, The Starr Foundation, Tipping Point Community, The Wallace 
Foundation, and the Weingart Foundation. The Wallace Foundation also provided additional 
support separate from their involvement with the Social Innovation Fund. 

Dissemination of MDRC publications is supported by the following funders that help finance 
MDRC’s public policy outreach and expanding efforts to communicate the results and implica- 
tions of our work to policymakers, practitioners, and others: The An nie E. Casey Foundation, The 
Harry and Jeanette Weinberg Foundation, Inc., The Kresge Foundation, Laura and John Arnold 
Foundation, Sandler Foundation, and The Starr Foundation. 

In addition, earnings from the MDRC Endowment help sustain our dissemination efforts. Contrib- 
utors to the MDRC Endowment include Alcoa Foundation, The Ambrose Monell Foundation, 
An heuser-Busch Foundation, Bristol-Myers Squibb Foundation, Charles Stewart Mott Founda- 
tion, Ford Foundation, The George Gund Foundation, The Grable Foundation, The Lizabeth and 
Fra nk Newman Charitable Foundation, The New York Times Company Foundation, Jan Nichol- 
son, Paul H. O’Neill Charitable Foundation, John S. Reed, Sandler Foundation, and The Stupski 
Family Fund, as well as other individual contributors. 


Opinions or points of view expressed in this document are those of the authors and do not neces- 
sarily reflect the official position of — or a position that is endorsed by — the CNCS, the Social 
Innovation Fund program, or other MDRC funders. 


Cover photo courtesy of Communities In Schools. 


For infomiation about MDRC and copies of our publications, see our website: www.mdrc.org. 
Copyright © 2015 by MDRC®. All rights reserved. 



Overview 


Too many students drop out and never earn their high school diploma. For students at risk of 
dropping out, academic, social, and other supports may help. Communities In Schools seeks to 
organize and provide these supports to at-risk students in the nation’s poorest-performing schools, 
including through “case-managed” services. School-based Communities In Schools site coordinators 
identify at-risk students, work with them individually to assess their needs, develop a case plan to 
meet those needs, connect them with supports in the school and community based on that plan, and 
monitor their progress to ensure that their needs are met. 

This report, the first of two from a random assignment evaluation of Communities In Schools case 
management, focuses primarily on the implementation of case management in 28 secondary schools 
during the 2012-2013 school year. The implementation research yielded several key findings: 

• The services provided by Communities In Schools were an important component of the 
participating schools’ support systems for students, but there were also many services provided 
by school staff members and other external partners. 

• Over about 30 weeks, case-managed students received an average of 19 service contacts totaling 
16 hours. More than 75 percent of case-managed students received academic services, about 60 
percent received social or life skills support, and half received behavior support. 

• “Higher-risk” case-managed students — those who failed a course or were chronically absent or 
suspended in the previous year — did not receive more case-managed services than others. 

• Compared with those randomly assigned to the non-case-managed group, case-managed 
students reported participating in more in-school support activities in several categories, includ- 
ing academically and behaviorally focused meetings with adults and mentoring. 

The report also includes interim one-year findings about case management’s impact on student 
outcomes. 

• Case management had a positive impact on students’ reports of having caring, supportive 
relationships with adults outside of home and school, the quality of their friendships, and their 
belief that education matters for their future. But for most outcomes concerning students’ inter- 
personal relationships and educational perspectives — relationships with caring, supportive 
adults at home or school and educational attitudes, engagement, goals, and expectations — there 
were no notable differences between case-managed and non-case-managed students. 

• After one year, Communities In Schools case management has not yet demonstrated improve- 
ment in students’ attendance or course performance, or reduced behaviors that lead to discipli- 
nary action — outcomes associated with increasing their chances of graduation. It is possible 
that case management could take more than a year to show an effect. 

This report concludes with suggestions for improvement for Communities In Schools based mainly 
on the implementation findings. The next report will present two-year impact findings and more 
about the implementation of case management in the 2013-2014 school year. 
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Preface 


School success for young people depends on more than teaching and learning; many students 
require supports that go beyond the classroom. Young people growing up in low-income commu- 
nities often face social and economic challenges that put obstacles in the way of their progress 
through school and increase the chances that they will drop out. Fortunately, schools and local 
community-based organizations often provide services to students intended to address these chal- 
lenges. Unfortunately, having an array of providers can result in fragmented or inconsistent service 
delivery, redundancies, and gaps, which can limit the power of these services to help students. 

Communities In Schools, an organization with a wide national reach, supports the im- 
plementation of a school-based model of integrated student support services that is intended to 
organize and supplement disparate services in a given school and community in an effort to 
strengthen their effectiveness. With a network of local affiliate offices across more than half the 
states in the nation, Communities In Schools has the potential to make a difference in the school 
outcomes of millions of students connected to its programming. Communities In Schools has 
also committed to being a learning organization, regularly evaluating aspects of its program- 
ming in order to improve its work on behalf of students. 

This report is the first of two from an experimental evaluation of Communities In 
Schools student case management being conducted by MDRC. Case management is one 
component of the Communities In Schools whole-school model. This evaluation is valuable for 
a few reasons. First of all, it is generating infomiation about aspects of Communities In 
Schools’ on-the-ground program implementation intended to encourage organizational reflec- 
tion and change regarding how to serve students better. Second, it is assessing the impact of 
case management, providing Communities In Schools staff members with information about 
how much of a difference they are making in the lives of case-managed students and where they 
might want to consider adjustments. Third, these findings will benefit other service providers 
who work with students facing similar obstacles. 

Targeting services is a central challenge for school-based student support programs. 
Schools located in low-income communities often serve large populations of students who are 
struggling academically in the face of other demands on their time and attention. In this context, 
should student support programs concentrate their resources on a smaller group of students 
facing the greatest obstacles to high school completion or on a larger group at somewhat lesser 
risk of dropping out? As one of the largest providers of school-based integrated student services, 
Communities In Schools encounters such dilemmas regularly. In this evaluation the researchers 
hope to inform this and other decisions for Communities In Schools and for the field at large. 

Gordon L. Berlin 

President, MDRC 
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Executive Summary 


Every day more than 7,000 students drop out of school. 1 One-fifth of students who enter high 
school do not graduate within four years, 2 and more than two-fifths of Latino and African- 
American boys drop out. 3 Compared with high school graduates, dropouts are more likely to 
live in poverty, suffer from poor health, be involved in crime, or be dependent on social ser- 
vices. 4 Many students at risk of dropping out need academic, social, and other supports to make 
it through high school, but these services are scattered across numerous government agencies 
and nonprofits. This fragmented delivery of services limits their potential to change the path of 
an at-risk student. And teachers and principals, for their part, are often overwhelmed by the 
emotional, social, and personal issues facing students. Integrating student support services and 
connecting them with schools is viewed as a promising approach to assist school staff members 
and help students stay on track to graduate. 5 

The Communities In Schools Model of Integrated 
Student Supports 

Communities In Schools uses an integrated student support model to assist schools and com- 
munities, working with low-income students at risk of failing or dropping out of the nation’s 
poorest-performing schools. Founded in 1977 by children’s advocate Bill Milliken, Communi- 
ties In Schools now serves more than 1.3 million students and their families. 6 It is active in over 
2,000 school sites, and the national office oversees a network of nearly 200 local affiliates in 26 
states and the District of Columbia. Communities In Schools’ national office establishes 
guidance on standards of practice, offers technical assistance to the local affiliates, and acts as a 
political advocate on behalf of the network. In these roles, the national office can influence how 
the affiliates approach model implementation within the local school districts with which they 


’Christopher B. Swanson, “Progress Postponed,” Education Week 29, no. 34 (2010): 22-23, 30. 

2 Richard J. Mumane, U.S. High School Graduation Rates: Patterns and Explanations (Cambridge, MA: 
National Bureau of Economic Research, 2013); Marie C. Stetser and Robert Stillwell, Public High School 
Four-Year On-Time Graduation Rates and Event Dropout Rates: School Years 2010-11 and 2011-12 
(Washington, DC: National Center for Education Statistics, U.S. Department of Education, 2014). 

3 Schott Foundation for Public Education, The Urgency of Now: The Schott 50 State Report on Public Ed- 
ucation and Black Males (Cambridge, MA: Schott Foundation for Public Education, 2012). 

4 Child Trends, “High School Dropout Rates,” last modified October 2014, http://www.childtrends.org/ 
?indicators=high-school-dropout-rates. 

5 Kristin A. Moore, Selma Caal, Rachel Carney, Laura Lippman, Weilin Li, Katherine Muenks, David 
Murphey, Dan Princiotta, Alysha Ramirez, Angela Rojas, Renee Ryberg, Hannah Schmitz, Brandon Stratford, 
and Mary Terzian, Making the Grade: Assessing Evidence for Integrated Student Supports (Bethesda, MD: 
Child Trends, 2014). 

Communities In Schools, 2013 Annual Report (Arlington, VA: Communities In Schools, 2014). 
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work. But as independent nonprofit organizations with their own boards of directors and local 
funding support, the affiliates have final determination in how the Communities In Schools 
model is carried out in their school sites. This autonomy results in variation in the details of the 
model’s implementation around the country. 

Nonetheless, there is a common understanding of the nature of the Communities In 
Schools comprehensive service model across the national network. The intent of this model is to 
reduce dropout rates by integrating community and school-based support services within 
schools through the provision of “Level 1” and “Level 2” services. Level 1 services are broadly 
available to all students or to groups of students and are usually short-term, low-intensity 
activities or services (for example, making clothing or school supplies available to students, 
organizing a school-wide career fair, or hosting a financial aid workshop for twelfth-graders). 
Level 1 services also include short-term “crisis” interventions when an extreme event disrupts a 
student’s life (for example, finding a solution if the power is turned off at the student’s home or 
providing short-term counseling in response to a traumatic event). Communities In Schools site 
coordinators — those responsible for all school-based operations — spend much of their time 
focused on more intensive Level 2 “case-managed” services, which they provide to a subset of 
students displaying one or more significant risk factors, such as poor academic performance, a 
high absentee rate, or behavioral problems. In case management, site coordinators work with 
individual students to identify their needs, connect them with supports in the school and 
community to address those needs, and regularly monitor their progress to ensure that their 
needs continue to be met. 

Communities In Schools Case Management 

This report focuses on the Level 2 case management component of the Communities In Schools 
comprehensive model of integrated student supports, as implemented at both the middle school 
and high school levels. 7 Figure ES.l presents the case management logic model. The “Con- 
text/Resources” column in the figure shows factors that support or affect case management 
service provision. For example, available financial resources and the number of students in a 
school influence how many site coordinators might be assigned there; existing youth and family 
service organizations represent the pool of potential local service providers with which site 
coordinators can partner. 

The second column, labeled “Activities,” outlines case management itself. Through a 
review of data or by referrals from adults in the school, the site coordinator identifies a student 


7 The comprehensive Communities In Schools model is implemented across grades K-12, in elementary, 
middle, and high schools. This evaluation of case management focuses only on secondary schools. 
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Evaluation of Communities In Schools 


Figure ES.l 


Communities In Schools (CIS) Case Management Logic Model 


Context/Resources 



Data 

• School and district 
characteristics 

• School needs 

• Student characteristics 

• Student needs 


Activities 

1 . At-risk student identified for case 
management 

2. Assessment completed to identify 
specific needs 

3. Case plan developed, includes 
services prescribed to address student 
needs 

4. Student receives services that fall 
into eight categories: 

• Academic services 

• Behavior services 

• Attendance services 

• Social/life skills services 

• Basic needs/resources 

• College/career preparation 

• Enrichment/motivation 

• Family-related services 

Service providers vary from school to 
school, but may include CIS site 
staff, a school staff member, and/or 
someone from outside the school, 
either a volunteer or a paid 
service provider 

5. CIS site staff monitor and adjust 
services based on student needs 


Mediators 


Student connection 
to caring adult 


W School engagement 


Attitudes about 
education 


Educational self- 
perception 


Outcomes 


Primary 

• Decreased dropout 
(decreased chronic 
absenteeism) 

• Increased high school 
graduation (reduced 
course failure) 

Secondary 

• Increased credit 
accumulation 

• Improved attendance 

• Decreased behavioral 
infractions 

• Improved academic 
performance 










as potentially at risk for eventually dropping out and seeks consent from a parent or guardian 
for the student to receive case-managed services. The site coordinator then assesses the 
student’s needs, develops an individualized case plan, and sets goals with the student. Based 
on the case plan, the site coordinator provides or connects the student to services specific to 
his or her needs. (See Box ES.l for examples of these services.) During the year, the site 
coordinator monitors student progress and adjusts the plan as necessary based on changes in 
the student’s needs. 

The case management activities are expected to affect “Mediators” (the third column) 
related to students’ attitudes, behaviors, and relationships. The services provided to a student are 
intended to help foster supportive relationships with adults and peers, encourage greater 
engagement with school, stimulate greater effort to meet academic and behavioral expectations, 
and increase the value that students see in their schooling. Impacts on these mediators are 
theorized to affect such student outcomes as attendance, perfonnance in class, and disruptive 
behavior in school, as listed in the “Outcomes” column. 


Box ES.l 

What Kinds of Activities Are Included in Each Service Type? 

Academic services. Adult or peer tutoring, homework assistance, study skills activities, 
student-teacher conferences 

Behavior services. Conflict resolution groups, anger management or other behavioral counsel- 
ing, violence prevention activities, behavior monitoring and interventions 

Attendance services. In-person attendance check-ins and planning 

Social or life skills services. Goal-setting activities; self-esteem enhancement activities; girls’ 
or boys’ groups; social, relationship, and communication activities; team-building games and 
activities; crisis and grief counseling services 

Basic needs and resources. Provision of school supplies; assistance with utilities, rent, etc.; 
food and clothing assistance; health activities and checkups 

College and career preparation. College admissions preparation and assistance, career 
counseling, college visits and career field trips, college awareness activities and programs 

Enrichment or motivation services. Community service, field trips unrelated to college or 
career preparation, sports or exercise activities, scouting, arts and crafts, student recognition 
activities and incentives 

Family-related services. Parent education, home visits, parent conferences and contacts, 
parent and family events and activities, family counseling 


ESA 





Evaluating the Communities In Schools Integrated 
Student Services Model 

In its ongoing commitment to continuous improvement, the Communities In Schools national 
office looks to external organizations to provide independent and objective research intended to 
help its staff understand how its model is being implemented in schools and what its impact is 
on schools and students. A previous evaluation by ICF International suggested that young 
people who receive Communities In Schools services are more likely to achieve a number of 
positive outcomes than those who do not. 8 Given the opportunity to expand its evidence base 
and strengthen its network through a federal grant program, Communities In Schools engaged 
MDRC, a nonprofit, nonpartisan education and social policy research organization, to conduct 
an independent, two-study evaluation funded by the Edna McConnell Clark Foundation Social 
Innovation Fund and the Wallace Foundation. 9 

One study focuses on the implementation and impact of Communities In Schools 
Level 2 case-managed services, examining service provision, student experiences, and student 
outcomes in the 2012-2013 and 2013-2014 school years. This study relies on a random 
assignment research design, often referred to as the “gold standard” evaluation design. 
Random assignment is a lottery-like process by which individuals are assigned either to 
participate in a specific program or to continue with whatever the “business as usual” altema- 
tive(s) might be. When there are more individuals interested in and eligible for a program than 
there are available slots, this process both provides a fair way to determine who participates in 
the program and creates two equivalent groups. The 16 middle schools and 12 high schools 
included in this study each had more eligible students — those facing academic, attendance, 
behavioral, and/or personal challenges that threatened to impede their progress toward high 
school graduation — than could be included on site coordinators’ caseloads. Thus students 
were randomly assigned to join site coordinators’ caseloads (1,140 students in the case- 
managed group) or to continue with business as usual at their schools, with access to whatev- 
er other student supports were available (1,090 students in the non-case-managed group). 
Since random assignment created two comparable groups and the sample is large, individual 
characteristics of the students are, on average, the same for both groups. Therefore, any 


S ICF International, Communities In Schools National Evaluation: Five Year Summary Report (Fairfax, 
VA: ICF International, 2010); Allan Porowski and Aikaterini Passa, “The Effect of Communities In Schools 
on High School Dropout and Graduation Rates: Results From a Multiyear, School-Level Quasi-Experimental 
Study,” Journal of Education for Students Placed at Risk (JESPAR) 16, no. 1 (2011): 24-37. 

9 The Edna McConnell Clark Foundation (EMCF) received a Social Innovation Fund (SIF) grant from the 
federal Corporation for National and Community Service. Communities In Schools is a subgrantee to EMCF 
within the SIF program. Thus, while Communities In Schools was interested in ongoing evaluation, this 
evaluation is also being conducted as one of the required activities of the SIF grant program. It also aligns with 
EMCF’s interest in supporting organizations that are participating in evidence-generating research. 
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differences that emerge over time between these two groups in outcomes such as their 
attendance, course performance, and behavior, as well as their attitudes about school and their 
relationships with peers and adults, can be attributed to Communities In Schools case man- 
agement. This report is the first of two planned for this study. 

The second study in this evaluation investigates the impact of the Communities In 
Schools comprehensive model, including both Level 1 and Level 2 services. In the second 
study, the impact of the comprehensive model is estimated by looking at student outcomes at 
schools before and after they implement the model and comparing the outcomes with those of 
students at a set of similar schools not implementing the model during the same period of 
time. 10 The results from this quasi-experimental study will be presented in a separate evaluation 
report. 

Studying the Implementation and Impact of Case Management 

This first report from the evaluation of Communities In Schools case management focuses 
primarily on its implementation. It looks at Communities In Schools’ operations and the 
contexts of the schools in which site coordinators work, the case management process, and how 
case management affects students’ school experiences, in particular which kinds of services 
students receive in an effort to help them succeed. Since it is expected that case management 
may take more than one year to start having an impact on students’ school outcomes, the one- 
year impact findings are considered interim findings. 11 The second report from this study of 
case management will present more definitive, two-year impact findings. 

This study draws upon varied quantitative and qualitative data sources to leam about 
case management’s implementation and interim impacts. They include surveys of school 
leaders and Communities In Schools site coordinators; in-person interviews with school 
principals, site coordinators, case-managed and non-case-managed students, and staff members 
of the local Communities In Schools affiliates; management infonnation system (MIS) data 
(regularly reported infonnation on the services site coordinators provide or coordinate for 


10 This is known as a “comparative interrupted time series” design. 

"Research on other student support programs such as AVID and the Higher Achievement Program have 
found null or negative impacts on outcomes such as attendance, course grades, and standardized test scores 
after the first year of student participation, and then found positive impacts by the second or third year of 
support. A report on integrated student services has also indicated that the impacts of such services can take 
time to emerge. See Elizabeth Dunn, Heather S. Fowler, Doug Tattrie, Claudia Nicholson, Saul Schwartz, 
Judith Hutchison, Isaac Kwakye, Reuben Ford, and Sabina Dobrer, BC AVID Pilot Project: Interim Impacts 
Report (Ottawa: Social Research and Demonstration Corporation, 2010); Carla Herrera, Jean B. Grossman, 
and Leigh L. Linden, Staying On Track: Testing Higher Achievement’s Long-Term Impact on Academic 
Outcomes and High School Choice (New York: A Public/Private Ventures project distributed by MDRC, 
2013); Moore etal. (2014). 
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individual students); surveys of case-managed and non-case-managed students; and student 
records data obtained from the local school districts. The first four data sources — adult 
surveys, in-person interviews, MIS data, and student surveys — all inform the implementation 
research, providing infonnation about the “Context” and “Activities” categories in the case 
management logic model. Student surveys also provide infonnation for the analysis of the 
impact of case management on mediating outcomes (“Mediators”). Student records data 
provide infonnation for the analysis of its impact on school outcomes (“Outcomes”). 

One-Year Implementation Findings 

The implementation research investigates the nature of Communities In Schools site 
coordinators’ work with students and adults within their schools — the “where, what, and how” 
of their work. The findings from this research address questions about context, fidelity of 
implementation, service receipt, and service contrast. 

Context: Where and Under What Circumstances Is Case Management Being 

Implemented? 

The 28 schools participating in this study are spread across seven school districts and 
five Communities In Schools local affiliates in North Carolina, South Carolina, and Texas. 
Most of these schools are located in and around large or midsized cities, and all receive funding 
under Title I, the federal program that supports schools that have a large proportion of disadvan- 
taged children. About 60 percent of the students in the study schools are eligible for free or 
reduced-price lunch, and nearly 85 percent are black or Hispanic students. On average, the high 
school and middle school student enrollments are more than 1,500 students and 700 students 
respectively. 

School leaders and Communities In Schools site coordinators reported that many types 
of services are offered by school staff, Communities In Schools, and other external partners to 
address challenges faced by their students. Communities In Schools appears to be an important 
part of the study schools’ student support environments, offering a variety of Level 1 services 
that are widely available to students in the school and providing Level 2 case-managed services 
to approximately 10 percent of students on average. The site coordinators in the study schools 
had an average of more than five years of experience in their positions and many had or were 
working toward advanced degrees in social work or counseling. 

Program Fidelity: How Similar Is Case Management As Implemented to the 

Model As Designed? 

Fidelity was assessed qualitatively, drawing predominantly on interviews conducted 
with site coordinators, school staff, and students on site visits conducted at 18 of the study 


ES-7 



schools across all five local affiliates. This assessment focused on the process of case manage- 
ment (Figure ES.2), or how site coordinators identified and worked with students on their 
caseloads. Each step of the case management process — identification, assessment, case 
planning, service provision, and monitoring and adjusting — was implemented across all visited 
sites, which is notable given the autonomy of the local affiliates within the Communities In 
Schools national network. Details of how each step was implemented varied across affiliates, 
however. 

Site coordinators primarily turn to administrators, teachers, and other support staff to 
help identify students in need of case management and to begin to understand students’ areas of 
need. But needs assessments for case-managed students are conducted differently across 
affiliates, with some being substantially more in depth than others. After the needs assessment, 
all site coordinators develop case plans for and with their case-managed students. The level of 
detail included in the case plan and the extent to which it guides practice throughout the school 
year also vary by affiliate and by school. 

Across schools and affiliates, the services provided for case-managed students focus 
primarily on academic assistance, behavior, and social skills development. Many services are 
provided directly by Communities In Schools staff or associated partner organizations; other 
services are activities or supports provided by school staff, which the site coordinator facilitates 
or encourages students to participate in. Once students have started receiving services, most site 
coordinators monitor case-managed students’ progress by reviewing students’ school records, 
although the frequency of these reviews differs across affiliates, and many site coordinators 
explained that fonnal adjustments to case plans may only occur periodically. 

Service Receipt: What Services Do Students Receive, How Many Times, and 

for How Long? 

Students in the Communities In Schools case-managed group received an average of 
19.4 service contacts during the year, which amounted to an average of 16.2 total Level 2 
service hours. These services include both those provided directly by site coordinators and those 
to which site coordinators referred students. Students were enrolled on caseloads for an average 
of about 30 weeks of the year, translating to about 2.5 service contacts monthly lasting an 
average of about 50 minutes each. The greatest proportion of case-managed students (three out 
of four) received academic services, three out of five received social or life skills services, and 
half received behavior-related services. 

There was substantial variation in service receipt among case-managed students, with 
some students receiving very low levels of services and others receiving high levels. The study 
team looked at whether this variation was associated with a student being “high risk” (having 
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Figure ES.2 

Communities In Schools Case Management Process 
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failed a course, been chronically absent, or been suspended in the prior year). 12 While there 
were minor differences in the levels of services Communities In Schools provided to high- and 
moderate-risk students, high-risk case-managed students did not receive more Level 2 service 
contacts or total hours overall than moderate-risk students. 13 If high-risk students, who are more 
likely to drop out of school, need more support than moderate-risk students, the provision of 
Level 2 services may need to be weighted more toward this group. 

Service Contrast: Does Case Management Create a Difference Between the 

Experiences of Case-Managed Students and the Experiences of Non-Case- 

Managed Students? 

Compared with non-case-managed students, Communities In Schools case-managed 
students generally reported participating in more support activities. Specifically, case-managed 
students were significantly more likely to report participating in individual and group meetings 
with adults in school, meeting with a mentor, participating in community service, and participat- 
ing in positive behavior programs. The two groups of students were similar in their receipt of 
such services as homework help, tutoring, and college and career planning, and their participa- 
tion in job shadowing or internships. At this stage of the study, it is unclear whether the number 
and magnitude of the differences between the two groups are enough to affect student outcomes. 
Also, because Communities In Schools coordinates Level 1 services accessible to all students, 
non-case-managed students have opportunities to engage with some of the same support services 
accessed by case-managed students. Thus it is possible that there is less contrast between the 
services used by the two groups of students than if Communities In Schools were not providing 
whole-school services as well as case management. 

Interim Impact Findings 

The implementation of Level 2 case management is intended to advance the larger goal 
of Communities In Schools to have a positive impact on students’ school progress. That is, 
Communities In Schools seeks to reduce the number of dropouts and to increase the number of 
graduates. During the time frame for this study, it is not possible to track all students through 
high school graduation. Therefore, the focus of the impact analysis is on primary outcomes that 


12 Balfanz, Herzog, and Maclver (2007) have found that as early as the sixth grade, 50 percent of future 
school dropouts in high-poverty schools exhibit indicators of falling off track — poor attendance, poor 
behavior, and poor course performance (that is, course failure). See Robert Balfanz, Liza Herzog, and Douglas 
Maclver, “Preventing Student Disengagement and Keeping Students on the Graduation Path in Urban Middle- 
Grades Schools: Early Identification and Effective Interventions,” Educational Psychologist 42, no. 4 (2007): 
223-235. 

13 Since all students must be deemed to have some risk to be eligible for case management to begin with, 
case-managed students who were not high risk are categorized as “moderate risk.” 
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are predictive of students dropping out: chronic absenteeism and course failure in core academic 
subject areas. 14 The study also analyzes the impact of case management on a few secondary 
outcomes', attendance rate, course grades, credit earning (in high school), and suspensions. 
Furthermore, the study looks at the impact of case management on mediating student outcomes 
related to school engagement, relationships with adults and peers, student self-perception, and 
educational aspirations and expectations, which often represent nonacademic obstacles to 
academic success for students at risk of dropping out. Communities In Schools seeks to support 
students in overcoming these obstacles, setting a foundation for them to succeed in school. (See 
“Mediators” and “Outcomes” in Figure ES. 1.) 

Primary and secondary outcomes. Compared with non-case-managed students, case- 
managed students had a slightly higher rate of chronic absenteeism and a similar rate of core 
course failure. There were no significant differences between the groups on other measures of 
school progress, behavior, and academic achievement. Thus, after one year, Communities In 
Schools case management has not yet demonstrated improved outcomes for students related to 
attendance, course performance, and school discipline. 

Mediating outcomes. Based on students’ reports, Communities In Schools case man- 
agement had a positive and statistically significant impact on students’ likelihood of having 
caring, supportive relationships with adults outside of home and school; on the quality of their 
peer relationships; and on their belief that education has positive value for their lives. But for 
most of the mediating outcomes — relationships with caring, supportive adults at home or 
school, educational attitudes, school engagement, and educational goals and expectations — 
there were no notable differences between students in the case-managed and non-case- 
managed groups. 

Suggestions for Continuous Improvement 

Although the evaluation study of Communities In Schools Level 2 case management is ongo- 
ing, the research after one year suggests some areas where Communities In Schools may want 
to consider change. 


l4 Allensworth and Easton (2005) indicate that earning course credits and not failing core courses in ninth 
grade is predictive of eventual graduation, and Herlihy and Kemple (2004) and Quint (2006) discuss how 
crucial ninth grade is to students’ progress to graduation. See Elaine M. Allensworth and John Q. Easton, The 
On-Track Indicator as a Predictor of High School Graduation (Chicago: Consortium on Chicago School 
Research, 2005); Corrine Herlihy and James J. Kemple, The Talent Development High School Model: Context, 
Components, and Initial Impacts on Ninth-Grade Students’ Engagement and Performance (New York: 
MDRC, 2004); Janet Quint, Meeting Five Critical Challenges of High School Reform: Lessons from Research 
on Three Reform Models (New York: MDRC, 2006). 
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• The implementation research suggests that the participating schools and affil- 
iates follow the same steps in the case management process — a noteworthy 
finding given Communities In Schools’ expansive national network of rela- 
tively autonomous local affiliates — but with variations. In particular, site 
coordinators’ ongoing assessment of students’ needs and their use of data 
collected as part of the monitoring process may benefit from greater con- 
sistency across schools. Communities In Schools’ national office could con- 
sider suggesting best practices for the network regarding how and with what 
kind of data site coordinators assess students’ ongoing needs, as well as how 
site coordinators can best use these data to ensure that services are adjusted to 
continually address students’ needs and increase the likelihood of improve- 
ments in student outcomes. 

• Recognizing that some students have more intensive needs than others, 
Communities In Schools may want to develop additional guidelines regard- 
ing the relationship between levels of service and student needs. Even 
though site coordinators indicated in interviews that they paid different lev- 
els of attention to different students according to their needs, the analyses 
of service receipt in this study showed little variation in the services re- 
ceived by high-risk and moderate-risk students. Site coordinators might 
benefit from more guidance on assessing levels of student risk and identify- 
ing appropriate levels of service in response, in order to focus more time 
and energy on the most struggling students. Such service differentiation, if 
standardized within the Communities In Schools model, would in effect 
result in a three-level service model. 

• The schools in this study have a range of services in place to help students be 
more successful, including the broadly available Level 1 services provided 
by Communities In Schools. In schools with many services available, it may 
be more challenging for Level 2 case management to make a difference 
above and beyond the other services that exist, including Communities In 
Schools’ own Level 1 services. Therefore, it may be beneficial for Commu- 
nities In Schools to consider where it can add the greatest value in each 
school building, and how that may change over time. Perhaps in schools with 
many services already available to students, Communities In Schools should 
focus their efforts on providing Level 2 case-managed services only to the 
students most in need and focus much less, if at all, on Level 1 . In schools 
with relatively few school-wide supports, the organization may be able to 
add substantial value by having site coordinators spend more time on Level 1 
services. The Communities In Schools national office may be in a position to 
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provide guidance to affiliates regarding how to determine an appropriate bal- 
ance between Level 1 and Level 2 services so that Communities In Schools 
can maximize its value in each school. 

Next Steps for the Study of Case Management 

The research activities that are part of this study have continued, and the results from ongoing 
analyses will be shared in a second report. The next report will build on and complement this 
report in three ways: 

• Two-year impacts. Similar student data on primary, secondary, and medi- 
ating outcomes is being collected for the 2013-2014 school year. The anal- 
ysis of these data will result in two-year impact findings that better assess 
the effectiveness of case management, given that most students on a site 
coordinator’s caseload receive case-managed services for more than one 
year. Preliminary data suggest that about two-thirds of the 2012-2013 case- 
managed students in our sample continued to receive case management in 
2013-2014. 

• Additional implementation findings. The second report will include more 
implementation infonnation, creating an opportunity to see whether the case 
management experience for students changed over the course of two years. It 
will again include information about service provision and receipt, as well as 
on the contrast that case management creates in the services students receive. 

New implementation data were collected during the 2013-2014 school year 
that will make it possible to discuss how the types of services provided to 
students align with their specific needs. The second report will also discuss 
the roles of Communities In Schools’ community partners and the nature of 
the partnerships. And to further understand the contrast that site coordinator 
case management might make in student service provision, the report will 
present more infonnation about the work of guidance counselors and social 
workers, the school staff members whose work is generally most comparable 
to that of Communities In Schools site coordinators. 

• Variation. Furthennore, the next report will investigate variation across 
school sites in tenns of both implementation and impacts and the associations 
between the two. This analysis may generate lessons about which contexts 
and implementation characteristics are associated with positive impacts on 
student outcomes. 
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Taken together, the two evaluation reports will provide comprehensive information 
about the implementation and impact of Level 2 case management. This information will be 
useful to Communities In Schools and other organizations that are trying to improve student 
outcomes through individualized case planning intended to better connect students to support 
services aligned with their needs. 
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Chapter 1 

Introduction 


Every day more than 7,000 students drop out of school. 1 Among Latino and African-American 
boys, the dropout rates are 42 percent and 48 percent, respectively. 2 Even though high school 
graduation rates have risen this century, too many students who enter public high school (one in 
five) do not graduate within four years. 3 Compared with high school graduates, dropouts are 
more likely to live in poverty, suffer from poor health, be involved in crime, or be dependent on 
social services. 4 Students considered at risk of dropping out are those facing the greatest 
challenges to earning their diplomas and in greatest need of academic, social, and other supports 
to make it through high school to graduation. Support services for these students are scattered 
across government agencies and nonprofits in communities with low-performing schools — 
fragmentation that often results in students receiving specific services in isolation, limiting the 
potential of any service to change the course of a student who is headed toward dropping out. 
Within schools, teachers and principals are often overwhelmed by the emotional, social, and 
personal issues facing students. Integrating student support services (that is, connecting com- 
munity agencies and organizations with schools and coordinating the services provided to 
students) is viewed as a promising approach to provide necessary assistance to school staff 
members and help keep students on track to graduate. 5 

The Communities In Schools Model of Integrated 
Student Supports 

Communities In Schools provides an integrated student support model to schools and communi- 
ties. Founded in 1977 by children’s advocate Bill Milliken, Communities In Schools works with 
low-income students at risk of failing or dropping out of the nation’s poorest-perfonning 
schools. The organization has extensive national reach and now serves more than 1.3 million 
students and their families. 6 It is active in over 2,000 school sites, and the national office 
oversees 17 state offices, as well as a network of nearly 200 local affiliates (independent 
nonprofit organizations) in 26 states and the District of Columbia. 7 The national office is 

'Swanson (2010). 

2 Schott Foundation for Public Education (2012). 

3 Mumane (2013); Stetser and Stillwell (2014). 

4 Child Trends (2014). 

5 Moore et al. (2014). 

Communities In Schools (2014). 

Communities In Schools (2014). 
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responsible for developing and enhancing the Communities In School model; communicating 
with national audiences, including advocating for education reform inclusive of integrated 
student services; fostering collaboration across the network; supporting research and evaluation 
of the model; and establishing national partnerships intended to generate resources and funding 
for members of the network. The local affiliates, each of which has a board of directors, oversee 
the implementation of the model in schools by site coordinators and build community partner- 
ships, developing local funding and resources to support the program. 8 

Communities In Schools has created a comprehensive service model that seeks to 
reduce dropout rates by integrating community-based supports within schools through both 
preventive “Level 1” services, which are available to all students in the school, and intensive, 
targeted, and sustained “Level 2” services for students who are displaying one or more signifi- 
cant risk factors, such as poor academic perfonnance, a high absentee rate, or behavioral 
problems. The implementation of Level 1 and Level 2 services at a school is led by a Communi- 
ties In Schools site coordinator, sometimes assisted by additional Communities In Schools staff 
members as case managers. 9 The site coordinator engages in yearly school-level needs assess- 
ments and planning with school leadership, teachers, and other members of the staff. The 
differences between the two service levels can be described in tenns of accessibility and 
duration or intensity. Level 1 services are broadly accessible and usually consist of short-term, 
low-intensity activities or assistance that students usually pursue voluntarily (for example, 
making clothing or school supplies available to students and hosting school-wide events). 10 
They also include short-term “crisis” interventions when an extreme event disrupts a student’s 
life (for example, finding a solution if the power is turned off at the student’s home or providing 
short-term counseling in response to a traumatic event). Level 2 services are targeted for 
specific students and are typically longer term and high intensity, delivered through a case 
management process that includes individualized assessments, goals, and plans. This process is 
managed by Communities In Schools school-based site coordinators, who seek to connect the 
students on their caseloads with services that address the specific challenges that each one faces. 


8 ICF International (2010). 

9 This report refers to the Communities In Schools staff members who provide case management as “site 
coordinators,” but their actual titles vary across schools. They may be known as case managers, success 
coordinators, program managers, or support specialists, or by other titles. 

IH While Level 1 services are school-wide, some are not truly available to all students because they target a 
broad subgroup, such as a college financial aid application workshop open only to twelfth-graders. Also, 
although discussed here primarily in terms of student services, Level 1 may also include services for parents. 
These are intended to strengthen students’ home supports and to more hilly engage parents in order to improve 
overall school climate. 
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Communities In Schools Case Management 

The focus of this report and the evaluation study discussed herein is the Level 2 case manage- 
ment component of the Communities In Schools comprehensive model of integrated student 
supports, as implemented at both the middle school and high school levels. 11 The ultimate goal 
of case management is to support students at risk of dropping out of high school. These risks 
emerge as early as the sixth grade, creating a need to serve students starting at least in middle 
school. 12 

While the specifics of how site coordinators manage student cases are discussed in 
detail in Chapter 3, this section presents a model for how case management is expected to 
influence student outcomes. Figure 1.1 is a visual representation of this case management logic 
model. The “Context/Resources” column in the figure shows the kinds of resources and 
contextual factors that support or influence the case management work. For example, available 
financial resources and the number of students in a school influence how many site coordinators 
might be assigned there. And site coordinators look to the community’s existing organizations 
that provide youth services (such as counseling, recreation, or jobs) to build partnerships and 
foster student opportunities. 

The second column, labeled “Activities,” outlines case management itself. The site 
coordinator identifies a student as potentially at risk for eventually dropping out, through a 
review of data or a referral from another adult in the school, and begins the process of adding 
the student to his or her caseload, first seeking consent from the parent(s) or legal guardian(s) 
for the student to receive case-managed services. The site coordinator assesses the student’s 
needs, develops an individualized case plan, and sets goals with the student, then provides or 
connects the student to the appropriate services. During the year, the site coordinator monitors 
student progress and adjusts the plan as necessary as the student’s needs change. Each step of 
this process is described in further detail in Chapter 3. 

Case management activities are expected to affect “Mediators” (the third column in 
Figure 1.1) related to students’ attitudes, behaviors, and relationships. The services provided to 
a case-managed student are intended to help foster supportive relationships with adults and 
peers, encourage greater engagement with school, stimulate greater effort to meet academic and 
behavioral expectations, and increase the value that the student sees in his or her secondary 
schooling. Impacts on these mediating outcomes are expected to lead eventually to changes in 


u The comprehensive Communities In Schools model is implemented across grades K-12, in elementary, 
middle, and high schools. This evaluation of case management focuses only on secondary schools. 

12 Balfanz, Herzog, and Maclver (2007). 
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Evaluation of Communities In Schools 


Figure 1.1 


Communities In Schools (CIS) Case Management Logic Model 


Context/Resources 



Data 

• School and district 
characteristics 

• School needs 

• Student characteristics 

• Student needs 


Activities 

1 . At-risk student identified for case 
management 

2. Assessment completed to identify 
specific needs 

3. Case plan developed, includes 
services prescribed to address student 
needs 

4. Student receives services that fall 
into eight categories: 

• Academic services 

• Behavior services 

• Attendance services 

• Social/life skills services 

• Basic needs/resources 

• College/career preparation 

• Enrichment/motivation 

• Family-related services 

Service providers vary from school to 
school, but may include CIS site 
staff, a school staff member, and/or 
someone from outside the school, 
either a volunteer or a paid 
service provider 

5. CIS site staff monitor and adjust 
services based on student needs 


Mediators 


Student connection 
to caring adult 


School engagement 


Attitudes about 
education 


Educational self- 
perception 


Outcomes 


Primary 

• Decreased dropout 
(decreased chronic 
absenteeism) 

• Increased high school 
graduation (reduced 
course failure) 

Secondary 

• Increased credit 
accumulation 

• Improved attendance 

• Decreased behavioral 
infractions 

• Improved academic 
performance 









traditional student performance measures such as attendance, perfonnance in class, and behav- 
ior in school that requires a disciplinary response, as listed in the final column, “Outcomes.” 


Evaluating the Communities In Schools Integrated 
Student Services Model 

In its ongoing commitment to continuous improvement, the Communities In Schools national 
office looks to external organizations to provide independent and objective research intended to 
help its staff understand how its model is being implemented in schools and what its impact is 
on schools and students. The national office, with funding from Atlantic Philanthropies, 
previously commissioned ICF International to conduct a five-year evaluation of its comprehen- 
sive model and the case management component. 13 The findings from that evaluation suggested 
that young people who receive Communities In Schools services are more likely to achieve a 
number of positive outcomes than those who do not. 14 Seeking to strengthen this evidence base, 
Communities In Schools welcomed further external evaluation of its comprehensive model and 
the case management component. 15 Accordingly, MDRC, a nonprofit, nonpartisan education 
and social policy research organization, is conducting an independent, two-study evaluation of 
Communities In Schools. This evaluation is supported primarily by the federal Social Innova- 
tion Fund (SIF). (See Box 1.1 for details about the SIF.) 

This report is the first of two reports planned for a study of the implementation and im- 
pact of Communities In Schools case management. This study relies on a random assigmnent 
research design, often referred to as the “gold standard” of evaluation designs. Random assign- 
ment is a lottery-like process by which individuals are assigned either to participate in a specific 
program or else to continue with whatever the “business as usual” altemative(s) might be. In 
cases where there are more individuals interested in and eligible for the program than there are 
available slots, this process provides a fair way to determine who participates in the program 
and also creates conditions for two equivalent groups. That is, the characteristics of individuals 
assigned to participate in the program should be the same as those of the individuals assigned to 
continue with business as usual. By comparing the outcomes of these groups, it is possible to 


13 ICF International (2010); Porowski and Passa (2011). 

14 Porowski and Passa (2011). 

15 The Edna McConnell Clark Foundation (EMCF) received a Social Innovation Fund (SIF) grant from the 
federal Corporation for National and Community Service. Communities In Schools is a subgrantee to EMCF 
within the SIF program. Thus, while Communities In Schools was interested in ongoing evaluation, this 
evaluation is being conducted as one of the required activities of the SIF grant program. It also aligns with 
EMCF’s interest in supporting organizations that are participating in evidence-generating research. In addition, 
this evaluation is supported by funds from the Wallace Foundation. 
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Box 1.1 


The EMCF Social Innovation Fund 

The Social Innovation Fund (S1F), an initiative enacted under the Edward M. Kennedy Serve 
America Act, targets millions of dollars in public-private funds to expand effective solutions 
across three issue areas: economic opportunity, healthy futures, and youth development and 
school support. This work seeks to create a catalog of proven approaches that can be replicated 
in communities across the country. The S1F generates a 3:1 private-public match, sets a high 
standard for evidence, empowers communities to identify and drive solutions to address social 
problems, and creates an incentive for grant-making organizations to target funding more 
effectively to promising programs. Administered by the federal Coiporation for National and 
Community Service (CNCS), the S1F is part of the government’s broader agenda to redefine 
how evidence, innovation, service, and public-private cooperation can be used to tackle urgent 
social challenges. 

The Edna McConnell Clark Foundation (EMCF), in collaboration with MDRC and The 
Bridgespan Group, is leading a S1F project that aims to expand the pool of organizations with 
proven programs that can help low-income young people make the transition to productive 
adulthood. The project is particularly focused on young people who are at greatest risk of 
failing or dropping out of school or of not finding work, who are involved or likely to become 
involved in the foster care or juvenile justice system, or who are engaging in risky behavior such 
as criminal activity or teenage pregnancy. 

EMCF, with its partners MDRC and Bridgespan, selected an initial cohort of nine programs 
and a second cohort of three programs to receive S1F grants: BELL (Building Educated 
Leaders for Life), Center for Employment Opportunities, Children’s Aid Society-Carrera 
Adolescent Pregnancy Prevention Program, Children’s Home Society of North Carolina, 
Communities In Schools, Gateway to College Network, PACE Center for Girls, Reading 
Partners, The SEED Foundation, WINGS for Kids, Youth Guidance, and Children’s Institute, 
Inc. These organizations were selected through a competitive selection process based on prior 
evidence of impacts on economically disadvantaged young people; a track record of serving 
young people in communities of need; strong leadership and a potential for growth; and the 
financial and operational capabilities necessary to expand to a large scale. 

The EMCF Social Innovation Fund initiative, called the True North Fund, includes support 
from CNCS and 15 private co-investors: The Edna McConnell Clark Foundation, The An nie 
E. Casey Foundation, The Duke Endowment, The William and Flora Hewlett Foundation, The 
JPB Foundation, George Kaiser Family Foundation, The Kresge Foundation, Open Society 
Foundations, The Penzance Foundation, The Samberg Family Foundation, The Charles and 
Lynn Schusterman Family Foundation, The Starr Foundation, Tipping Point Community, The 
Wallace Foundation, and the Weingart Foundation. 
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assess the causal impact of the program, because any differences between them after random 
assignment can be attributed to the program. 

As will be detailed later in the report, the schools included in the study serve predomi- 
nantly low-income and minority students, and each had more students eligible for case man- 
agement than could be served by Communities In Schools site coordinators. (Eligible students 
face academic, attendance, behavioral, and/or personal challenges that threaten to impede their 
progress toward high school graduation.) Thus students were randomly assigned to join site 
coordinators’ caseloads (the case-managed group) or to continue with business as usual at their 
schools, with access to whatever student supports were available other than Communities In 
Schools case management (the non-case-managed group). Any differences that emerge over 
time between these two groups of students in their attendance, course performance, and behav- 
ior, as well as their attitudes about school and their relationships with peers and adults, can thus 
be attributed to Communities In Schools case management. 

MDRC’s evaluation also includes a complementary second study that investigates the 
impact of the Communities In Schools comprehensive model, inclusive of both Level 1 and 
Level 2 services. This second study uses a quasi-experimental comparative interrupted time 
series design, in which the impact of the comprehensive model is estimated by looking at 
student outcomes at schools before and after they implement the model and comparing them to 
student outcomes over the same period at a set of similar schools that are not implementing the 
model. The quasi-experimental study uses a different set of schools from those included in the 
random assigmnent study. The results from the quasi-experimental study will be presented in a 
separate evaluation report. 


Learning Agenda for the Study of the Implementation and 
Impact of Case Management 

The goals of this study are to provide evidence of the causal impact of Communities In Schools 
case management on student outcomes and to understand the details about how case manage- 
ment aims to support students and improve their school trajectories. Specifically, the random 
assigmnent evaluation design examines the incremental effect of Communities In Schools’ 
Level 2 case management on student outcomes, particularly progress toward graduation, over 
and above the effect of Level 1 and other services provided to all students in the study schools. 
The implementation research looks at the details of the case management process, helps shed 
light on the impact findings, and provides useful information to Communities In Schools about 
the case management component of its model. 

The primary focus of this first report from this multiyear evaluation is on the implemen- 
tation of case management. Three of the remaining four report chapters focus on Communities 
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In Schools’ operations and the contexts of the schools in which site coordinators work, the case 
management process, and how case management affects students’ school experiences — in 
particular, what kinds of services students receive that are intended to help them succeed 
in school. 

Since it is expected that case management may take more than one year to start having 
an impact on students’ school outcomes, the one-year impact findings are considered interim 
findings. Research on other student support programs has found null or negative impacts on 
outcomes such as attendance, course grades, and standardized test scores after the first year of 
student participation, and then found positive impacts by the second or third year of support. For 
example, the research organization SRDC found that the AVID educational pilot project had 
negative impacts on attendance and course grades for ninth-grade students in British Columbia, 
Canada, but by eleventh grade these negative impacts had disappeared and the AVID students 
were doing better in their classes and taking more rigorous coursework than their peers in a 
control group . 16 Herrera, Grossman, and Linden reported that the Higher Achievement Pro- 
gram’s after-school and summer programs did not affect the math and reading perfonnance of 
students after one year but had statistically significant positive impacts on both subjects after the 
second year . 17 In addition, in their report on integrated student services, Moore et al. indicated 
that the effects of such services can take time to emerge . 18 Still, improvement after one year on 
at least some of the mediating student outcomes measured in this study would suggest possible 
later impacts on how students do in school. The second report from this study of case manage- 
ment will present more definitive two-year impact findings. 

The Implementation of Case Management 

The case management implementation research provides an opportunity to discuss how 
Communities In Schools site coordinators work with students and adults within their assigned 
schools. Given that the Communities In Schools model represents an effort to integrate multiple 
types of school and community services, it is necessary to consider the local contexts in which 
the model is implemented. The implementation research addresses questions related to four 
topics: context, model fidelity and program quality, amount and duration of services provided, 
and service contrast. Research questions related to each of these topics are as follows: 

• Context. What are the circumstances under which case management is being 
implemented? 


16 Dunn et al. (2010). 

17 Herrera, Grossman, and Linden (2013). 
18 Moore et al. (2014). 



• Model fidelity and program quality. How similar is case management as 
implemented to the model as designed? How well does it align with the 
needs of students? 

• Amount and duration of services. What services do students receive, how 
many times, and for how long? 

• Service contrast. Does case management create a difference between the 
experiences of case-managed students and the experiences of non-case- 
managed students? 

The Impact of Case Management 

Although Communities In Schools hopes to have an impact on multiple student out- 
comes, the driving mission of the organization is to affect students’ school progress — that is, 
to reduce the number of students who drop out and increase the number of students who 
graduate. During the time frame for this study, it is not possible to track all students through 
high school graduation. Therefore, the two key impact questions focus on primary outcomes 
that are suggestive or predictive of students dropping out: chronic absenteeism and course 
failure in core subject areas f 

Students need to attend school and pass their required courses to make progress toward 
graduation. Site coordinators do not provide direct content instruction, although they will 
connect students to academic services, such as tutoring, intended to help them leam course 
material. Successfully passing a course, however, requires not only that students understand the 
material, but also that they complete their homework, behave appropriately in class, and attend 
on a regular basis, all of which are targets of case management. These key questions will be 
examined for all students in the study (middle school and high school): 

Impact questions. Does case management reduce the number of students 

1. who are chronically absent (attendance below 90 percent)? 

2. who fail one or more of their core academic classes? 


l9 Allensworth and Easton (2005) indicate that earning course credits and not failing core courses in ninth 
grade is predictive of eventual graduation, and Herlihy and Kemple (2004) and Quint (2006) discuss how 
crucial ninth grade is to students’ progress to graduation. Balfanz, Herzog, and Maclver (2007) have also found 
that as early as the sixth grade, 50 percent of future school dropouts in high-poverty schools exhibit indicators 
of falling off track — poor attendance, poor behavior, and poor course performance (that is, course failure). 


9 



The study will also analyze the impact of case management on a few secondary out- 
comes : attendance rate, course grades, high school credit earning, and suspensions. Further- 
more, the study will look at the impact of case management on the mediating student outcomes 
discussed above, such as students’ relationships with peers and adults, their engagement with 
school, how much they value education, how they see themselves as students, and their educa- 
tional goals. (See “Mediators” and “Outcomes” in Figure 1.1.) 

Data Sources 

This study draws upon varied quantitative and qualitative data sources to leam about the 
implementation of Communities In Schools case management and to analyze the impact of case 
management on student outcomes. These data provide information that corresponds to the 
categories presented in the case management logic model: context, activities, mediators, and 
outcomes (Figure 1.1). The first four data sources — adult surveys, in-person interviews, 
management information system (MIS) data, and student surveys — are all part of the imple- 
mentation research. 20 Student surveys also provide information for the analysis of the impact of 
case management on mediating outcomes. The last data source — student records data — 
provides information for the analysis of the impact of Level 2 case management on the primary 
and secondary school outcomes. 

Adult surveys. In the spring and summer of 2013, the study team administered surveys 
to school leaders and Communities In Schools site coordinators at the 28 participating schools. 21 
The school leader surveys and the site coordinator surveys provide information about school 
and community context, such as what kinds of needs students had; what kinds of support 
programs and services were available to all students and whether they were provided by school 
staff members, Communities In Schools staff members or partners, or other organizations; and 
what issues the respondents saw as important to the school. In addition, the site coordinators 
reported information specific to their work with case-managed students, such as those students’ 
needs and what services site coordinators provided to try to support them. (See “Context” and 
“Activities” in Figure 1.1.) 

In-person interviews. Also in the spring of 2013, the study team visited 18 of the 28 
school sites participating in the evaluation across all five local affiliates. The main focus of these 


20 The full text of all MDRC surveys administered during the first year of the evaluation are included in 
supplementary Appendix C in Corrin et al. (2015), available on MDRC’s website (www.mdrc.org). 

2 'in a majority of the study schools, the school leader survey was completed by the principal or assistant 
principal, but some cases it was completed by another staff member identified by the principal as knowledge- 
able about student support services at the school (for example, the head of guidance, an academic dean, a 
guidance counselor, or a social worker). One school leader survey was completed per school. 
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visits was to conduct in-person interviews with Communities In Schools site coordinators, 
school principals, case-managed and non-case-managed students, and Communities In Schools 
affiliate staff members. The data collected from all of these interviews provide information 
about the local implementation contexts. In addition, interviews with the site coordinators gave 
valuable information about how they describe the process of case management and how they 
practice it. The interviews with the case-managed students provided some illustrative examples 
of how they experience case management. Through the site visits, the study team was also able 
to do informal observation of the schools and, in some schools, observe an activity provided or 
coordinated by Communities In Schools site coordinators. 

Management information system (MIS) data. MIS data are used in the analysis of 
the implementation of case management to measure the amount and types of services case- 
managed students received. Depending on the local affiliate, these data came from Communi- 
ties In Schools’ national infonnation system or from similar state or local systems. Throughout 
the year site coordinators enter information into these systems related to the services they 
provide or coordinate for individual students on their caseloads, making it possible to analyze 
the amount and types of services these students receive. As described in Chapter 4, the study 
team organized these services into eight categories: academic, behavior, attendance, social or 
life skills, basic needs and resources, college and career preparation, enrichment or motivation, 
and family-related services. (See “Activities” in Figure 1.1.) 

Student surveys. Students in both study groups — case-managed and non-case- 
managed — responded to baseline surveys in the fall of 2012, before case management service 
provision began, and follow-up surveys in the spring of 2013. The baseline surveys provide 
infonnation about student characteristics, such as whether older siblings graduated high school 
or dropped out, students’ household composition, and the educational background of their 
parents. These data allow for additional description of the study sample and are also included in 
the analysis of the comparability of case-managed and non-case-managed students. 22 On the 
follow-up surveys, students reported on what kinds of supports they received in and out of 
school, allowing the study team to compare support services received by case-managed and 
non-case-managed students. In addition, these surveys provide infonnation about mediating 
outcomes, such as the students’ engagement with school, their relationships with peers and 
adults, and educational aspirations. (See “Mediators” in Figure 1.1.) Both the baseline and 
follow-up surveys make use of items from the California Healthy Kids Middle School Survey 
(specifically items from Module A and resilience items from Module B) with pennission of the 
California Department of Education and WestEd. 


22 These baseline data also provide co variates in the impact analyses of survey outcomes. 
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Student records data. To answer the primary and secondary research questions about 
the impact of case management on students’ school outcomes, the study relies on student 
records data obtained from the participating school districts. These districts provided baseline 
data (pre -random assignment) about students for the 2011-2012 school year and follow-up data 
(post-random assignment) for the 2012-2013 school year. Both sets of data included infor- 
mation about students’ attendance, course performance, and suspensions, as well as student 
demographic characteristics, such as gender, race/ethnicity, and English language learner 
status. The baseline data make it possible to describe the sample of students in the study and 
analyze the comparability of the case-managed students and their non-case-managed peers at 
the start of the study. 23 The follow-up data provide the necessary information to determine 
whether case management had an impact on measures related to attendance (such as chronic 
absenteeism and attendance rate), course perfonnance (such as course failure, average grades, 
and credit earning in core courses), and behavior (such as number of suspensions). (See 
“Outcomes” in Figure 1.1.) 


Structure of the Report 

This report focuses on the implementation of case management. Thus the next three chapters 
cover implementation context, the case management process, and service provision. Chapter 2 
describes the types of schools participating in this study and the services provided to their 
students. Chapter 3 takes an in-depth look at the Communities In Schools case management 
process, discussing how site coordinators work with students on their caseloads. Chapter 4 
describes the student participants, looks at how much support case-managed students received, 
and also investigates whether and how case-managed and non-case-managed students differed 
in terms of the types and amount of support received. The final chapter goes beyond implemen- 
tation, presenting interim impact findings after one year of case management. It first discusses 
analyses of mediating outcomes, then turns to the results of analyses of the impact on attend- 
ance, behavior, course performance, and school progress. It concludes with a few programmatic 
considerations for Communities In Schools and other providers of integrated student services. 


“In addition, because a prior measure of a later outcome is a good predictor of that outcome, the baseline 
data are included as covariates in the impact analyses, helping to improve the precision of the impact estimates. 


12 



Chapter 2 


Study Schools’ Characteristics 
and Support Services 


This chapter of the report includes information about the Communities In Schools affiliates and 
schools included in this study. It begins by detailing the study affiliate and school selection 
processes and describing the characteristics of the participating schools. The chapter then 
presents information about the support services available in the study schools and about Com- 
munities In Schools’ site-level operations. This sets the stage for a detailed examination of the 
Communities In Schools case management process and services and an understanding of the 
different services received by case-managed and by non-case-managed students, which are 
explored in Chapters 3 and 4. 

Key points from this chapter include the following: 

• The study sample includes 28 schools supported by five Communities In 
Schools affiliates located in North Carolina, South Carolina, and Texas. All 
the study schools receive Title I funding and have many students facing chal- 
lenges related to their academics, attendance, behavior, and home lives. 

• At each school, one or more Communities In Schools site coordinators are 
responsible for all on-site operations. The site coordinators in the study had 
an average of more than five years of experience as site coordinators and 
many had or were pursuing advanced degrees in social work or counseling. 

• School leaders and Communities In Schools site coordinators reported that 
many types of services are offered by a school’s staff, Communities In 
Schools staff members and partners, and external partners to address stu- 
dents’ needs. Communities In Schools appears to be an important part of the 
support environment in study schools, offering a variety of Level 1 services 
to all students and providing more intensive Level 2 case-managed services 
to a subset of students in each school. 


Affiliates and Schools in the Study 

Twenty-eight schools supported by five Communities In Schools affiliates are included in this 
study. To build the study sample, the MDRC research team worked with Communities In 
Schools’ national office during the 201 1-2012 school year to recruit affiliates and schools. As 
the first step of the recruitment process, Communities In Schools staff members identified eight 
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affiliates that indicated openness to participating in the national evaluation and that had been 
previously determined to meet national quality standards. 1 

After the national office introduced MDRC and the evaluation to these affiliates, the 
MDRC research team conducted introductory phone calls with each affiliate to discuss the study 
procedures and explain the conditions necessary for conducting a student-level random assign- 
ment study. The research team sought affiliates with multiple schools in which the Communities 
In Schools staff believed there were at least twice as many students in need of Level 2 case 
management as could be served at the school. This condition was considered necessary for study 
participation because MDRC would randomly assign students to either receive or not receive 
case management only where Communities In Schools lacked the resources to serve all the 
students in need. When need exceeds available resources, random assignment is a fair way to 
allocate those scarce resources. Thus, as part of the initial screening process, affiliates estimated 
the number of open slots they expected to be available for new case-managed students during the 
2012-2013 school year. While site coordinators’ average caseload size in the study schools was 
84 students, the only slots available for random assignment would be those that would be newly 
open for the 2012-2013 school year (for example, spots left open by graduating eighth or twelfth 
graders). 2 The research team considered affiliates to be high-potential study participants if they 
(1) had multiple schools in which the Communities In Schools staff expected 20 or more open 
caseload slots, to ensure a large enough sample for the study, and (2) believed there to be twice 
as many students eligible for case management as could be served. 

After the introductory phone calls and initial screening were completed for the eight af- 
filiates, the research team made site visits to six high-potential affiliates and selected five to be 
included in the study. 3 Twenty-eight schools — 16 middle schools and 12 high schools — 
across the five affiliates were selected to participate in the study based on the criteria given 
above. One affiliate is in South Carolina, two are in North Carolina, and two are in Texas. Two 


Selected affiliates had been accredited through the national Communities In Schools Total Quality Sys- 
tem accreditation process. As part of the process, each affiliate was verified via formal review to have achieved 
the organization’s established business and site operations standards. 

2 The site coordinators in the study reported having an average of 84 students on their caseloads, but in 
most school years, a relatively small percentage of students are new to case management because site coordina- 
tors continue working with many of the students they were serving the year before. For example, a high school 
site coordinator might be able to serve 100 students on her caseload. At the start of a given year, she might 
continue working with 70 of the students who were on her caseload during the previous year, while 30 of her 
students from the previous year might have graduated or moved to another school. In this case, she would have 
room for 30 new students on her caseload, so 30 study students would have been randomly assigned to this site 
coordinator’s caseload. 

3 The sixth affiliate was not included in the study because the affiliate staff was not confident that there was 
oversubscription at each of the potential study schools. 
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Evaluation of Communities In Schools 


Table 2.1 


Selected Characteristics of the Study Schools, 
Schools in Study States, and the 
National Population of High Schools and Middle Schools 



Study 

Schools 

Schools in 
Study States 3 

National 

Schools' 1 

High schools 

School locale (%) 

Large or midsize city 

63.6 

12.4 

18.3 

Urban fringe and large town 

27.3 

18.0 

29.9 

Small town and rural area 

9.1 

69.6 

51.7 

Schools with Title I status (%) 

100.0 

74.8 

62.6 

Average school enrollment (#) 

1,527 

914 

775 

Average student enrollment (%) 

Grade 9 

29.9 

29.3 

24.7 

Grade 10 

26.5 

25.3 

24.5 

Grade 1 1 

22.1 

23.5 

24.4 

Grade 12 

21.5 

21.8 

25.9 

Students receiving free or reduced-price 

lunch (%) 

59.1 

50.9 

44.3 

Race/ethnicity of students (%) 

Black 

42.1 

32.4 

15.0 

Hispanic 

37.0 

8.4 

20.0 

White 

16.2 

53.5 

57.1 

Other 

3.6 

5.7 

7.9 

Female students (%) 

48.2 

49.3 

48.2 

Average number of full-time teachers 

86 

55 

49 

Number of schools 

11 

695 

16,278 


(continued) 


affiliates are located in large cities and three are in midsize cities, though some also serve a 
number of suburban and rural schools in adjacent areas. These affiliates have long-standing 
histories working with their local districts, all five with at least ten years of operation in their 
communities, and most with more than twenty. 

Each affiliate contributed between 2 and 13 schools and between 100 and 1,025 students 
to the study sample. Table 2.1 includes the characteristics of the study schools, as compared 
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Table 2.1 (continued) 



Study 

Schools 

Schools in 
Study States 3 

National 

Schools 3 

Middle schools 

School locale (%) 

Large or midsize city 

78.6 

14.0 

19.1 

Urban fringe and large town 

7.1 

23.5 

36.9 

Small town and rural area 

14.3 

62.5 

43.9 

Schools with Title I status (%) 

100.0 

84.1 

73.9 

Average school enrollment (#) 

737 

667 

625 

Average student enrollment (%) 

Grade 6 

34.3 

33.6 

32.7 

Grade 7 

32.6 

33.3 

33.5 

Grade 8 

33.1 

33.0 

33.6 

Students receiving free or reduced-price 

lunch (%) 

61.5 

57.3 

49.3 

Race/ethnicity of students (%) 

Black 

44.0 

31.0 

16.0 

Hispanic 

44.1 

10.6 

21.2 

White 

7.6 

52.2 

54.7 

Other 

4.3 

6.2 

8.1 

Female students (%) 

48.3 

48.7 

48.5 

Average number of full-time teachers 

48 

42 

40 

Number of schools 

14 

630 

9,981 


SOURCES: 2011-2012 data obtained from the U.S. Department of Education 
National Center for Education Statistics Common Core of Data (CCD), Public 
School Universe Data. 

NOTES: The study includes a total of 16 middle schools and 12 high schools. Of 
the 16 middle schools in the study, 2 schools were excluded from analysis due to 
their classification as elementary schools in the 2011-2012 CCD; data for grades 6- 
8 were not available for these two schools. One middle school in the study serves 
grades 6-12. To calculate average enrollment by grade for this school, grade 9 
enrollment was excluded from the denominator. Of the 12 high schools in the 
study, 1 high school was excluded from analysis because no CCD information was 
available for the school. 

Teacher counts in the CCD are reported in full-time equivalent units. This is the 
amount of time required to perform an assignment stated as a proportion of a full- 
time position. It is computed by dividing the amount of time an individual is 
employed by the time normally required for a full-time position. The counts were 
rounded to the nearest whole number. 

“National and study states samples include study schools. 
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with public schools in study states and across the country. More of the study schools are in 
urban areas, with more than two-thirds located in or around large or midsize cities. 4 Compared 
with average schools in their states and nationally, the study schools serve greater proportions of 
low-income and minority students; all the study schools receive funds under Title I, the federal 
program that supports schools that have a large proportion of disadvantaged children. On 
average approximately 60 percent of the students in these schools are eligible for free or 
reduced-price lunch, and nearly 85 percent are black or Hispanic students. The study high 
schools enroll an average of 1,527 students and the middle schools enroll an average of 737 
students, compared with respective national averages of 775 and 625 students. In addition, the 
study high schools have lower proportions of students in upper grades compared with schools 
nationally, which signals higher rates of dropout. 

Interviews with Communities In Schools site coordinators and school leaders in the 
study schools indicated that, in addition to serving large proportions of low-income and minori- 
ty students, these schools have student populations with substantial academic, behavioral, and 
personal needs. Site coordinators and school leaders at most schools explained during inter- 
views that numerous community and home-life challenges might hinder students’ success at 
school to varying degrees. Primary examples of the struggles they cited include living in 
neighborhoods with high poverty and limited resources, having limited transportation options, 
living in overcrowded homes, coming from single-parent households to which students are 
often expected to contribute financially, and becoming parents before finishing school. One site 
coordinator explained the difficulties faced by students in his school: 

Obviously, we’re in the inner city; there are a lot of needs for our students. Many 
are at risk for a bunch of different reasons. It could be gang involvement, high 
absenteeism, some of our students are pregnant. Just low achievement, low mo- 
tivation, but at the same time there are a lot of great kids here who go on to have 
great success. 

Similarly, many other site coordinators and school principals articulated concerns about 
such obstacles but also expressed optimism that, with the right support in school, many students 
would go on to experience college and career success. 

Table 2.2 includes additional information about the problems that students in the 
study schools encounter. School leaders and site coordinators indicated that poor academic 


4 A larger proportion of study schools are located in urban areas compared with other middle and high 
schools in Communities In Schools’ national network, as well. Information provided by the Communities In 
Schools national office, based on 2012-2013 end-of-year reports by all local sites and affiliates except those in 
Texas, indicated that 38 percent of network middle schools were located in urban areas, 21 percent in suburban 
areas, and 41 percent in rural areas. Of high schools in the network, 53 percent were located in urban areas, 14 
percent in suburban areas, and 33 percent in rural areas. 
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Evaluation of Communities In Schools 


Table 2.2 


Student Issues and Their Priority for School Leaders and Communities In Schools Site Coordinators 


Measure 

Poor 

Academic 

Performance 

Poor 

Attendance 

Behavior/ 

Discipline 

Problems 

High Risk 
Social 
Behavior 

Lack of Parental 
Involvement/ 
Support 

Family 

Instability 

Students at 
Risk of 
Dropping Out 11 

School leader estimate of percentage 
of students facing issue* 5 

52.4 

30.4 

33.2 

27.9 

49.6 

33.2 

30.4 

School leaders reporting issue is 
a high priority at school 1 (%) 

100.0 

79.2 

96.0 

76.2 d 

70.8 

54.2 

— 

Site coordinator estimate of 
percentage of students facing issue 

58.5 

34.8 

38.9 

38.1 

60.0 

40.8 

43.0 

Site coordinators reporting issue is a 
high priority for CIS at school (%) 

89.3 

70.8 

95.8 

68.2 e 

52.2 

56.5 

— 

Number of schools: school leader survey 

= 25; site coordinator survey = 

28 






SOURCES: School leader survey (summer 2013), site coordinator survey (summer 2013). 

NOTES: The percentages presented here include only those respondents who answered the survey item; response rates vary among items. Missing 
values were excluded from calculations. The rates of missing responses to the school leader survey items average to 2 percent and range from 0 
percent to 16 percent, while the rates of missing responses to the site coordinator survey items average to 8 percent and range from 0 percent to 19 
percent. CIS = Communities In Schools. 

a These values are calculated from an individual item on the school leader and site coordinator surveys, asking: “Of all students at your school, 
what percentage would you describe as being at risk of dropping out?” The percentages presented here are calculated from the mean of an 11- 
point answer scale, which ranges from 0 = “about 0%” to 10 = “about 100%.” 

b The percentages presented here are calculated from the mean of an 1 1-point answer scale, where respondents estimated the percentage of 
students in need at their school from 0 = “about 0%” to 10 = “about 100%.” 

c The survey item was skipped if respondents reported 0 percent of students faced the given issue, or if services relating to the issue were not 
offered at the school. 

d The missing rate for this item is 16 percent. 
e The missing rate for this item is 19 percent. 

Respondents at 3 of the 28 study sites did not return a survey. 



performance and lack of parental involvement and support affected the greatest number of 
students in their schools — at least half. In addition, they estimated that nearly 30 percent or 
more of their students had issues with poor attendance, behavioral or discipline problems, 
high-risk social behavior, or family instability. All or nearly all school leaders and site 
coordinators indicated that poor academic performance and behavioral or discipline problems 
were a high priority for their schools and for Communities In Schools. As shown in the final 
column of Table 2.2, school leaders’ and site coordinators’ estimates of the percentage of 
students at risk of dropping out were 30 percent and 43 percent, respectively. 5 


Services Available in the Study Schools 

In addition to learning about the study schools’ student populations, the research team sought to 
leam more about the availability of support services in the schools. Overall, the infonnation 
shared by school leaders and site coordinators suggests that the study schools offer a wide 
variety of services that aim to address students’ needs. 6 

Many school administrators explained during interviews that noninstructional supports 
are a critical aspect of how their schools serve students, with more than one-third of principals 
suggesting that providing behavioral and personal supports and resources was as important as, 
and sometimes more important than, serving students academically. In many cases a central 
team, consisting of both teachers and members of the support staff (for example, guidance 
counselors, social workers, and in some cases Communities In Schools site coordinators), 
confers regularly to match various services with needs across the overall student population. 
Some principals explained that these core teams are important forums for teachers and support 
staff members to discuss students’ challenges and plan appropriate support for those who 
struggle the most. 

The site coordinator and school leader surveys suggest that many services are available 
in the study schools. As shown in Table 2.3, site coordinators in most study schools reported 
that Communities In Schools provides services to address each of the issues listed, ranging from 
a high of 96 percent of site coordinators reporting that Communities In Schools staff members 
or external partners provide services to address poor academic perfomiance to a low of 81 
percent reporting that they provide services to address family instability. (Partners are people 
and organizations that Communities In Schools brings into the school or works with outside of 


5 On every one of the seven issues in the survey, site coordinators estimated a larger percentage of students 
affected than did school leaders. 

6 School leader and Communities In Schools site coordinator surveys were the primary data source for this 
information, as such data are not available consistently from the school districts participating in the study. 
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Evaluation of Communities In Schools 


Table 2.3 


Student Issues and Reported Service Providers 



School Leaders Reporting 
Service Provider (%) 

Site Coordinators Reporting 
Service Provider (%) 

Issue 

School 

Staff 

CIS Staff 
or Partners 

Other 

School 

Staff 

CIS Staff 
or Partners 

Other 

Poor academic performance 

96.0 

84.0 

56.0 

96.4 

96.4 

35.7 

Poor attendance 

92.0 

92.0 

44.0 

77.8 

88.9 

3.7 

Behavior/discipline problems 

100.0 

84.0 

44.0 

77.8 

85.2 

22.2 

High-risk social behavior 

100.0 

80.0 

40.0 

65.4 

92.3 

30.8 

Lack of parental 

involvement/ support 

92.0 

80.0 

44.0 

65.4 

88.5 

23.1 

Family instability 

92.0 

84.0 

56.0 

61.5 

80.8 

15.4 

Number of schools 



25 a— 



28 


SOURCES: School leader survey (summer 2013), site coordinator survey (summer 2013). 

NOTES: The values presented here represent the percentage of school leaders or site 
coordinators who reported each of the given service providers, and include only those surveys 
in which the respondent answered the survey item. Response rates vary among services. 
Missing values were excluded from calculations. The rate of missing responses to the school 
leader survey items is 0 percent. The rates of missing responses to the site coordinator survey 
items average to 5 percent and range from 0 to 7 percent. 

a Respondents at 3 of the 28 study sites did not return a survey. 


the school.) School leaders concurred, with at least 80 percent of leaders reporting that Commu- 
nities In Schools staff members or partners provided services to address each of the issues. This 
suggests that school leaders view Communities In Schools as an active collaborator in address- 
ing the needs of their school populations. 7 

While they were not asked about which school staff members provided each of the ser- 
vices, school leaders did provide information about support staff positions in their schools. 
Nearly all (96 percent) school leaders reported having guidance counselors, 64 percent reported 


7 While nearly all school leaders indicated that school staff members provide services to address each of 
these issues, site coordinators did not always agree. Almost all site coordinators indicated that school staff 
members provided services to address poor academic performance, but they were less likely to say so about the 
other issues. Site coordinators were also less likely than school leaders to indicate that other service providers 
addressed students’ issues. 
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having social workers, and 48 percent reported having other support staff members (cited 
examples include behavior specialist, college adviser, and parent advocate). In schools that had 
these support positions, school leaders indicated having an average of 2.8 full-time equivalent 
(FTE) guidance counselors, 1.0 FTE social worker, and 1.6 FTE other support staff members. 
Across all the schools in the sample, the average number of support staff members was 4.2 
FTEs, which did not include the Communities In Schools site staff. These school support staff 
members had large caseloads: guidance counselors, social workers, and other support staff 
members averaged 296 students, 419 students, and 257 students per FTE, respectively, accord- 
ing to school leaders. 

Table 2.4 details the types of support services school leaders reported being available in 
their schools, with their estimates of the percentage of students receiving each type of service, 
service frequency, whether each service is available to all, and whether Communities In Schools 
is a key provider. At least 80 percent of leaders reported the following services being available 
in their schools: academic assistance, mentoring, meetings with adult staff members to discuss 
academic goals, college planning and preparation, behavior intervention, anger management or 
conflict resolution, linkages to basic needs and resources, and individual and school-sponsored 
family engagement activities. In schools where the services are offered, leaders reported that 
approximately half of all students receive academic assistance, college planning and prepara- 
tion, and pregnancy prevention services, and more than half the students engage in meetings 
with adults to discuss academic goals, programs to prevent bullying and drug abuse, career 
development and readiness programs, and school activities or meetings with their families. In 
addition, many of these services are available on at least a monthly or near-weekly basis. School 
leaders also reported that Communities In Schools is a key provider for many of these services 
— most frequently for mentoring, numerous family-related services, behavior interventions, 
drug abuse prevention, community service activities, and linkages to basic needs and resources. 
In addition to their survey responses, most school leaders stated during interviews that they 
consider Communities In Schools to be an important part of their school’s support system. 


Communities In Schools’ Site-Level Operations 

As part of its implementation research, the study team sought to build an understanding of how 
Communities In Schools operates in the study schools. This section introduces details about 
staffing, space, and programming. 

Site Coordinators and Their Workspace 

Communities In Schools program implementation is led at each school by one or more 
site coordinators responsible for all on-site operations. Affiliate staff members determine how 
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Table 2.4 

Student Services As Reported by School Leaders 


Service 

Leaders 
Reporting Service 
Is Offered (%) 

Estimated Percentage 
of Students 
Receiving Service 3 

Leaders Reporting 
Service Available 
to All Students (%) 

Average Y early 
Service 
Frequency* 3 

Leaders Reporting 
CIS Is a Key Service 
Provider (%) 

Academic services 

Academic assistance 

92.0 

47.4 

95.7 

34.6 

52.2 

Meeting with adult staff to discuss 
academic goals 

80.0 

74.0 

NA 

16.2 

55.0 

Behavior and attendance services 

Behavior intervention 

80.0 

35.0 

NA 

24.3 

70.0 

Anger management/Conflict resolution 

80.0 

26.5 

NA 

22.6 

60.0 

Truancy prevention activities 

60.0 

38.7 

42.9 

18.8 

40.0 

Social/life skills services 

Mentoring 

84.0 

23.8 

47.6 

21.5 

81.0 

Gang intervention/prevention L 

56.0 

30.0 

50.0 

10.6 

42.9 

Pregnancy prevention 

40.0 

49.0 

30.0 

17.3 

40.0 

Bullying prevention 

72.0 

73.3 

83.3 

18.6 

61.1 

Drug abuse prevention 

56.0 

66.4 

76.9 

13.1 

85.7 

Substance abuse support programs 

44.0 

27.3 

NA 

9.6 

27.3 

Pregnancy/parent support programs 

44.0 

22.7 

NA 

6.2 

36.4 

Programs for adjudicated youth d 

28.0 

35.7 

NA 

14.6 

42.9 

Grief support programs 

56.0 

16.4 

NA 

8.6 

28.6 

Programs for LGBT youth 6 

16.0 

6.0 

NA 

9.0 

20.0 
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Table 2.4 (continued) 



Leaders 

Estimated Percentage 

Leaders Reporting 

Average Yearly 

Leaders Reporting 


Reporting Service 

of Students 

Service Available 

Service 

CIS Is a Key Service 

Service 

Is Offered (%) 

Receiving Service 3 

to All Students (%) 

Frequency 13 

Provider (%) 

Basic needs/resources 

Linkages to basic needs/resources 

80.0 

39.0 

79.0 

28.3 

80.0 

Physical health screening 

68.0 

38.8 

43.8 

5.2 

52.9 

Exercise class or club/Obesity 
intervention and prevention 

44.0 

36.4 

63.6 

29.3 

36.4 

College/career preparation 

College planning and preparation 

80.0 

50.0 

65.0 

17.9 

65.0 

Career development/readiness programs 

68.0 

58.2 

62.5 

20.2 

64.7 

Job shadowing or internship 

36.0 

20.0 

11.1 

13.8 

44.4 

Enrichment/motivation services 

Community service/Service learning 

52.0 

23.8 

30.8 

17.2 

76.9 

After- or before-school programs 

60.0 

38.0 

93.3 

31.6 

46.7 

Family-related services 

Individual family engagement activities 

80.0 

65.5 

94.7 

11.2 

80.0 

Parent group meetings and input activities 

76.0 

52.6 

94.4 

9.2 

89.5 

School-sponsored activities for students 
and their families 

80.0 

54.5 

89.5 

10.2 

75.0 

Number of schools 

25‘ 
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Table 2.4 (continued) 


SOURCE: School leader survey (summer 2013). 

NOTES: The values presented in the rightmost four columns include only those surveys in which the 
respondent reported the service was offered at the school and answered the relevant survey item; 
response rates vary among items. A value of NA indicates the question was not asked for this service 
on the school leader survey. Missing values were excluded from calculations. The rate of missing 
responses to the service availability item (column 3) averages to 3 percent and ranges from 0 percent to 
7 percent. The rate of missing responses to the service frequency item (column 4) averages to 12 
percent and ranges from 0 percent to 60 percent. The rate of missing responses to all other items is 0 
percent. 

a The percentages presented here are the mean of an 11 -point scale, where respondents estimated the 
percentage of students in need at their school from 0 = “about 0%” to 10 = “about 100%.” 

b The values presented here are approximations of the average number of times per year the service 
is offered. Times per year were calculated from respondents' answers to an original four-point 
frequency scale, where 1 = “once or twice a year” (1.5 times per year), 2 = "less than once a month” 

(4.5 times per year), 3 = “1-2 times a month” (13.5 times per year), and 4 = “one or more times a week” 
(36 times per year). 

c The missing rate for this service frequency item was 21 percent. 
d The missing rate for this service frequency item was 43 percent. 
e The missing rate for this service frequency item was 60 percent. 
f Respondents at 3 of the 28 study sites did not return a survey. 


many site coordinators to place in a school based on a variety of factors, including the size of 
the school, student needs, and available funding. 8 In each of the study schools, there was at least 
one full-time school-based Communities In Schools site coordinator, with half the study schools 
having two or more site coordinators. 9 

The site coordinators in these schools were relatively well experienced and well educat- 
ed. In the spring of 2013, site coordinators in the study schools had an average of 5.25 years of 
experience working as Communities In Schools site coordinators and 3.25 years of experience 
in that role in their current schools. In addition, over 60 percent of the site coordinators either 
had or were pursuing a graduate degree, with a majority focused on social work or counseling. 
Across the Communities In Schools national network, site coordinators have an average of 3.15 
years of experience in their role and 2.5 years of experience in their schools. 10 


8 In some cases, local affiliates also place other support people in schools to work with site coordinators. 
Some examples include AmeriCorps members and undergraduate or graduate-level student interns. 

fourteen of the study schools had one site coordinator, thirteen had two site coordinators, and one had 
three site coordinators. 

l0 This information was provided by the Communities In Schools national office, based on 2012-2013 end- 
of-year reports from all local sites and affiliates except those in Texas. 
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Schools generally provide the site coordinators with separate space to carry out program 
operations. In all but one of the study schools visited by the research team, they had dedicated 
workspace — many site coordinators had their own classroom or small building, others shared 
classrooms with another support person (for example, the parent coordinator), and others had 
small private offices near other support staff members. The Communities In Schools space in 
the majority of schools visited had substantial amounts of student activity, with frequent 
scheduled and unscheduled interactions occurring between students and Communities In 
Schools staff members throughout the day. When their offices were housed in somewhat 
disconnected areas of the schools that saw less student traffic (for example, portable buildings 
set back from the main school building), site coordinators seemed to experience fewer student 
interactions. 

Communities In Schools Services 

Communities In Schools site coordinators are involved in a wide variety of activities in 
their schools. While they report to work every day at their assigned schools and engage in 
yearly school-level needs assessments and planning activities with school leaders, site coordina- 
tors are employed and managed by their local Communities In Schools affiliate rather than the 
schools themselves. Communities In Schools staff members explained that site coordinators are 
provided with guidelines from the affiliate about how to do their work but have substantial 
autonomy for deciding how to carry out the specific tasks. 

As part of the comprehensive model, Communities In Schools provides both “Level 1” 
services, which are available to all students in the school, and more intensive, targeted, and 
sustained “Level 2” case management for students who are at greater risk of not succeeding in 
school. While there is some variation in how site coordinators spend their time, the primary 
emphasis across the study sites is on Level 2 case management. Site coordinators reported 
spending an average of 44 percent of their time in a typical week planning for or providing 
Level 2 services and an additional 14 percent of their time working with or managing partners 
who provide Level 2 services. During a typical week, site coordinators reported spending an 
average of 10 percent of their time planning or providing Level 1 services and 6 percent of their 
time working with or managing partners who provide Level 1 services. Another 19 percent of 
site coordinators’ time is spent working on Communities In Schools-related administrative 
tasks, such as record keeping and reporting. Interviews with site coordinators suggest that many 
of these administrative tasks involve monitoring their case-managed students, checking grades 
and attendance and completing progress reports. Finally, 7 percent of site coordinators’ time is 
spent tending to other matters, including school-related tasks that are outside of their Communi- 
ties In Schools responsibilities. During interviews, half of the site coordinators said that they 
have non-Communities In Schools duties, whether taken on voluntarily, assigned by school 
administrators, or requested by the school’s faculty or staff. 
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Level 1 services. While site coordinators reported spending a majority of their time in a 
typical week on Level 2 case management, the Level 1 services that are broadly available to all 
students or to large groups of students represent an important component of the Communities In 
Schools model. During interviews, site coordinators reported providing numerous Level 1 
services for the larger school population, including making clothing or school supplies available 
to students, conducting in-class presentations and workshops, leading school-wide service 
projects, bringing guest speakers into the school, and conducting college and career awareness 
activities. While site coordinators may spend comparatively less time on Level 1 events or 
services in a given week, some Level 1 services are set up by site coordinators near the begin- 
ning of the year and require minimal ongoing maintenance time after that (for example, partner- 
ships with outside organizations that come in to work with students after school). In addition to 
one-time events like speakers or college fairs that are broadly open to students, Level 1 services 
include short-term counseling or “crisis management” for students with pressing needs that arise 
unexpectedly. 

Table 2.5 includes site coordinators’ reports of Level 1 activities, frequency of service 
provision, and the extent to which non-case-managed students use Level 1 services. At least 
three-quarters of the site coordinators reported offering the following Level 1 services: college 
preparation and career development programs, behavior interventions, bullying prevention, 
links to basic needs and resources, and school-sponsored activities for students and their 
families. Many of these services were provided frequently; site coordinators reported that half 
of all Level 1 services were provided at least once or twice monthly. Although mentoring and 
before-school or after-school programs were offered as Level 1 services in only 46 and 32 
percent of study schools, respectively, those were the Level 1 services provided most frequently 
— on about a weekly basis — where they were available. 

Finally, underscoring school leaders’ indication that Communities In Schools is a key 
service provider in many areas, site coordinators reported that Level 1 services are used by 
students for whom they provide case management as well as by those who are not on their 
caseloads. The far right column in Table 2.5 includes an estimate of the proportion of case- 
managed students among those receiving Communities In Schools Level 1 services, with 0.5 
representing an even split between case-managed and non-case-managed students receiving a 
given service. Site coordinators reported that more than two-thirds of Level 1 service types 
were provided to a roughly even mix of case-managed and non-case-managed students. The 
remaining services, although open to non-case-managed students, had higher levels of case- 
managed than non-case-managed student participation. This aligns with site coordinators’ 
reports during interviews that they regularly encourage their case-managed students to 
participate in Level 1 activities. 


26 



Evaluation of Communities In Schools 


Table 2.5 


Level 1 Service Provision As Reported by Site Coordinators 


Service 

Site Coordinators Average 

Reporting Service Yearly Service 
Is Offered (%) Frequency 3 

Proportion of Students 
Receiving Services Who 
Are Case-Managed b 

Academic services 

Academic assistance 

Meeting with adult staff to discuss 

60.7 

27.4 

0.5 

academic goals 

64.3 

20.3 

0.7 

Behavior and attendance services 

Behavior intervention 

75.0 

20.3 

0.5 

Anger management/Conflict resolution 

67.9 

21.4 

0.5 

Truancy prevention activities 

39.3 

15.0 

0.6 

Social/life skills services 

Mentoring 

46.4 

34.3 

0.7 

Gang intervention/prevention 

50.0 

13.4 

0.5 

Pregnancy prevention 

42.9 

22.1 

0.8 

Bullying prevention 

75.0 

9.4 

0.5 

Drug abuse prevention 

57.1 

8.2 

0.6 

Substance abuse support programs 

39.3 

6.8 

0.6 

Pregnancy/parent support programs 

28.6 

14.3 

0.7 

Programs for adjudicated youth 

17.9 

8.7 

0.6 

Grief support programs 

32.1 

8.8 

0.5 

Programs for LGBT youth" 

17.9 

9.9 

0.3 

Basic needs/resources 

Linkages to basic needs/resources 

85.7 

19.1 

0.6 

Physical health screening 

Exercise class or club/Obesity intervention 

35.7 

9.9 

0.7 

and prevention" 

28.6 

13.5 

0.6 

College/career preparation 

College planning and preparation 

78.6 

15.7 

0.5 

Career development/readiness 

82.1 

13.7 

0.5 

Job shadowing or internship 

35.7 

4.2 

0.7 

Enrichment/motivation services 

Community service/Service learning 

53.6 

7.4 

0.7 

After- or before-school programs 

32.1 

33.5 

0.6 


(continued) 
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Table 2.5 (continued) 


Service 

Site Coordinators Average 

Reporting Service Y early Service 
Is Offered (%) Frequency 3 

Proportion of Students 
Receiving Services Who 
Are Case-Managed b 

Family-related services 

Individual family engagement activities 

60.7 

7.6 

0.8 

Parent group meetings and 
input activities 

67.9 

7.7 

0.6 

School-sponsored activities for students 
and their families 

75.0 

7.3 

0.5 

Number of schools 

28 




SOURCE: Site coordinator survey (summer 2013). 

NOTES: The values presented in the final two columns include only those surveys in which 
respondents reported the service was offered and answered the relevant survey items; response rates 
vary among items. Missing values were excluded from calculations. The rates of missing responses to 
the service frequency item (column 2) average to 1 percent and range from 0 percent to 13 percent. The 
rates of missing responses to the case-managed student proportion item (column 3) average to 4 percent 
and range from 0 percent to 25 percent. 

a The values presented here are approximations of the average number of times per year the service is 
offered. Times per year were calculated from respondents' answers to an original four-point frequency 
scale, where 1 = “once or twice a year” (1.5 times per year), 2 = “less than once a month” (4.5 times per 
year), 3 = “1-2 times a month” (13.5 times per year), and 4 = “one or more times a week” (36 times per 
year). 

b The values presented here are the means of a three-point scale, where 0 = “mostly non-case- 
managed students,” 0.5 = “an even mix of case-managed and non-case-managed students,” and 1 = 
“mostly CIS case-managed students.” CIS = Communities In Schools. 

c The missing rate for this case-managed student proportion item is 25 percent. 


Level 2 case-managed services. At each school site, Communities In Schools site 
coordinators provide Level 2 case-managed services to a subset of students. Across the Com- 
munities In Schools national network, Level 2 case management is provided to an average of 
approximately 10 percent of students in a school. 11 As discussed in detail in Chapter 3, the case 
management process involves a series of steps aimed at providing at-risk students with individ- 
ualized support to address their needs. The steps in the process include identifying students in 
need of case management, assessing individual students’ needs, developing a case plan and 
providing or comiecting students with services to address their needs, monitoring students’ 
progress throughout the year, and making adjustments to services students receive, as needed, 
so that they continue to meet students’ needs during the school year. 


1 'This information was provided by the Communities In Schools national office, based on 2012-2013 end- 
of-year reports from all local sites and affiliates except those in Texas. 


28 







Site coordinators do not always provide direct service to case-managed students; con- 
necting students with supports may involve making students aware of and helping them take 
advantage of existing support services available to all students in the school, such as after- 
school tutoring offered by teachers. Additionally, site coordinators may encourage case- 
managed students to participate in specific Level 1 services that they offer. However, site 
coordinators also provide services that are available specifically for case-managed students that 
go above and beyond what is otherwise available to students in the school, such as site coordi- 
nator-facilitated group activities or Communities In Schools mentoring or tutoring. While other 
students in the school may take advantage of some of the same services, Communities In 
Schools case-managed students have an adult in the school who is specifically tasked with 
supporting their individual needs and checking on them throughout the school year. 

Each site coordinator is responsible for providing Level 2 case management to a specif- 
ic group of students — the coordinator’s “caseload.” For schools with multiple site coordina- 
tors, the division of case-managed students varies by affiliate, with some caseloads split up 
based on grade level, gender, or student needs (for example, one site coordinator may work 
primarily with pregnant and parenting students), or simply based on availability on a site 
coordinator’s caseload (for example, one site coordinator may have had more graduating seniors 
leave open spots on her caseload and would, therefore, work with a greater proportion of the 
new students the following year). 

Site coordinators in the study reported on the survey that they had an average of 84 stu- 
dents on their caseloads. While site coordinators in a majority of the study schools reported this 
size as very similar to previous years, those in one-quarter of the schools reported that it was 
smaller. Reported caseload sizes varied both across and within affiliates in the study, with an 
overall range of 28 to 215 case-managed students per coordinator and average caseloads across 
the five affiliates of 44, 66, 77, 92, and 130. Caseload sizes are set for each school by the 
affiliate; in one affiliate, caseloads are kept intentionally small, while other affiliates set higher 
caseload maximums when site coordinators have assistance from other adults working for 
Communities In Schools, such as interns or AmeriCorps staff members. The next chapter 
discusses further variation in case management across the affiliates participating in the study. 


Summary 

This chapter of the report provided infomiation about the Communities In Schools affiliates and 
schools included in this study. The study schools primarily serve low-income and minority 
students and have many students facing challenges related to their academics, attendance, 
behavior, and home lives. These schools, however, seem to have fairly comprehensive student 
supports in place, according to Communities In Schools staff members and school leaders. 
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Communities In Schools appears to be an important part of the support services envi- 
ronment in the study schools, offering a variety of Level 1 services to all students and providing 
more intensive Level 2 case-managed services to an average of 10 percent of the students. 
While schools also had an average of more than four FTEs of other support staff (for example, 
guidance counselors and social workers), those school staff members had average caseloads 
more than three times as large as those of Communities In Schools site coordinators. This 
suggests that site coordinators should be able to provide more support to the students on their 
caseloads than other school support staff members can. Chapter 3 details site coordinators’ work 
with case-managed students, including how they identify students, assess their needs, develop 
plans for support services, connect or directly provide students with services, and monitor 
students’ progress. 
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Chapter 3 

Implementation of Case Management 


This chapter first provides a general overview of the Communities In Schools case management 
process, then describes each step of the process — student identification, needs assessment, case 
planning, service provision, and monitoring student progress and adjusting services. The 
subsequent discussion of the process draws on data from the school leader and site coordinator 
surveys, field interviews, and observations. This chapter primarily addresses the question of 
whether the steps of the case management process were implemented in the study schools. The 
research team did not develop quantitative “fidelity scores” to measure how well, or to what 
extent, sites implemented each step, as one important component of the Communities In 
Schools model is that local sites and affiliates have the freedom to adapt the model to their 
school contexts. The descriptions of each step of the case management process in this chapter 
include general themes that emerged from interview and survey data and infonnation about 
variation across the study sites. 

This chapter makes the following main points: 

• Each step of the case management process — identification, assessment, case 
planning, service provision, and monitoring and adjusting — was imple- 
mented across the 18 schools that the research team visited, though the 
details varied across affiliates. 

• Site coordinators primarily turn to administrators, teachers, and other support 
staff members to help identify students in need of case management and to 
begin to understand students’ areas of need. There is variation, however, 
across the affiliates in how needs assessments are conducted, with some 
being substantially more in depth than others. 

• Although all site coordinators develop case plans for and with their case- 
managed students, the level of detail included in the case plan and the extent 
to which it guides practice throughout the school year varies by affiliate and 
by school. 

• Across schools and affiliates, the services provided for case-managed stu- 
dents focus primarily on academic assistance, behavior, and social skill 
development. Many services are provided directly by Communities In 
Schools staff members or associated partner organizations; school staff 
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members also provide activities or supports, which the site coordinator 
facilitates or encourages students to participate in. 

• Most site coordinators monitor case-managed students’ progress by review- 
ing students’ school records, although the frequency of these reviews varies 
across affiliates, and many site coordinators explained that fonnal adjust- 
ments to case plans may only occur periodically. 


The Case Management Process 

Communities In Schools site coordinators and affiliate staff members described case manage- 
ment as providing individualized attention and guidance to students from their program case- 
loads. It may include developing trusting relationships with their case-managed students, 
providing and connecting students with support customized to each one’s needs, creating safe 
spaces for them to share their problems or express their views, and offering new frames of 
reference about the world outside of school. The majority of site coordinators and affiliate staff 
members also described case management as monitoring students’ attendance, academic 
progress, and general mental and emotional health. 1 

As depicted in Figure 3.1, the steps in the case management process include identifying 
students in need of case management, completing assessments to identify students’ needs and 
set goals, developing case plans to address these, connecting and/or directly providing students 
with case-managed services, and monitoring students’ progress and adjusting services as 
needed. Each of these steps is discussed in turn below. 

Step 1 : Student Identification 

The Communities In Schools case management process begins with the identification 
of at-risk students who would benefit from receiving additional support services. These students 
typically encounter multiple factors that jeopardize their ability to be in class or to perform to 
acceptable academic standards, making them more likely to fail or drop out. Communities In 
Schools targets these students to receive more personalized support from a site coordinator in 
order to reduce the chances of them dropping out. 


'in order to be consistent with terminology, a few guidelines are used to describe qualifiers that summarize 
the frequency of findings across the sites. Specifically, across the 18 sites that were visited, the use of “a few” 
pertains to 3 or 4 sites where a given finding emerged, “some” refers to 5 or 6 sites, “many” indicates 6 to 1 1 
sites, “most” indicates 12 to 14 sites, and “almost all” pertains to 15 or more sites. 
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Figure 3.1 

Communities In Schools Case Management Process 
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Site coordinators from the 18 schools visited described the process of identifying the 
students in need of case management with relative consistency, though the specific resources 
and personnel they used to identify these students varied. In almost all the study schools, site 
coordinators typically fill their caseloads through referrals and recommendations from teachers, 
administrators, guidance counselors, or other student support staff members on campus. 2 To 
encourage teachers and staff members to make these referrals, site coordinators from many 
schools described making presentations about Communities In Schools in student classes or 
during teacher meetings or trainings, as well as having individual discussions with teachers and 
support staff members. For instance, site coordinators from two schools explained that they 
schedule infonnational sessions during faculty meetings to provide an overview of Communi- 
ties In Schools and to walk teachers through the process of referring students whom they 
consider to be good candidates for case management. In some cases, principals facilitate the 
referral process by scheduling site coordinators to address teachers during in-service training 
sessions at the beginning of the school year. 

In addition to using teacher and staff referrals, some affiliates have their own or district- 
specific guidelines for identifying eligible students, which is where slight variations in the 
student identification process occur. One Communities In Schools affiliate, for example, has 
contractual agreements with such local agencies as the city housing authority, financial assis- 
tance programs, and the foster care system, which require site coordinators to enroll at least a 
certain number of students served by those agencies. 3 Staff members from another affiliate 
indicated that their school district provides a list of students considered to be “eligible” for 
Communities In Schools case management based on a number of at-risk factors, from which 
site coordinators identify candidates by reviewing attendance and grade records from the 
previous year(s). In some instances site coordinators revealed that students sometimes self- 
select into the program, mainly as a result of having heard presentations given by the site 
coordinators in their classes or by word of mouth from friends already enrolled in case man- 
agement. Regardless of how students are initially identified as eligible, consent from a parent or 
legal guardian is required for all students in order to be officially enrolled on site coordinator 
caseloads and to start receiving Level 2 services. 


2 This description refers to the years before the study and therefore does not include the random assignment 
procedure employed during the first year of the study. Please refer to the study sample description in Chapter 4 
for additional details about how students were identified for case management during the study year. 

3 These agencies serve the types of at-risk students they would be likely to enroll even if these contracts 
were not in place. 


34 



Step 2: Needs Assessment 

After identifying students to receive case management, site coordinators conduct indi- 
vidual assessments to leam more about their needs and to gather additional information about 
their backgrounds — the second step in Figure 3.1. This involves a more comprehensive review 
of student records and discussions with students, school staff members, and parents. 

Site coordinators at almost all schools explained that they conduct a review of school 
records data, such as attendance records, grades, and behavioral incident records, to better 
understand the patterns of need for their students. In schools across three different affiliates, 
most mentioned also relying on teachers to leam about their students’ behavior or academic 
progress and conducting classroom observations. 

While most site coordinators also reported using standardized assessment tools 
designed to measure levels and areas of need, these tools varied by affiliate. For example, site 
coordinators from one affiliate first conduct a full psychosocial assessment of each student, 
which helps them pinpoint the socio-emotional conditions and the external circumstances that 
may affect his or her behavior. Another affiliate has coordinators complete a detailed inventory 
of their students’ risk factors and protective factors. Site coordinators in some other affiliates, 
however, engage in less investigative needs assessments. Site coordinators from a few of the 
schools in these affiliates characterized the assessment as a more intuitive process that is mainly 
informed by the initial check-in meetings with their students. 

Site coordinators and staff members from most affiliates mentioned the importance of 
communicating with family members to identify and address student issues. One affiliate places 
a particularly strong emphasis on actively involving the family in the needs assessment process 
by visiting the homes of case-managed students or meeting with parents in school, or at least 
comiecting with parents on the phone to talk about their child’s needs. All the site coordinators 
in this affiliate reported visiting or at least trying to visit all their case-managed students’ homes. 
As one site coordinator explained, “I think that’s really the difference in what we do ... we go 
into the home and [we] meet mom or dad or grandma . . . and say ‘I’m meeting you on your 
ground because I really want to work with [your child] and I want them to be successful.’” 
Other site coordinators from this affiliate suggested that reaching out to parents and family 
members allows them to get a better understanding of how a student’s home circumstances 
affect their school perfonnance or behavior. 

On the survey, site coordinators shared information about the needs of their case- 
managed students. As shown in Table 3.1, site coordinators indicated that poor academic 
perfonnance is the issue faced by the greatest proportion of case-managed students (about 60 
percent). Over a quarter are assessed as having poor attendance (about 28 percent) or engaging 
in high-risk social behavior (about 31 percent), such as gang participation, drug use, and/or 
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Table 3.1 


Case-Managed Student Issues and Service Details As Reported by Site Coordinators 


Measure 

Poor 

Academic 

Performance 

Poor 

Attendance 

Behavior/ 

Discipline 

Problems 

High-Risk 

Social 

Behavior 

Lack of Parental 
Involvement/ 
Support 

Family 

Instability 

Respondents' estimate of percentage of 

case-managed students facing issue d 

58.2 

28.1 

40.7 

31.1 

48.5 

37.3 

Site coordinators reporting most or 

all students in need are served (%) 

78.6 

70.8 

91.7 

72.7 

65.2 

60.9 

Service quality rating 13 

0.7 

0.7 

0.8 

0.6 

0.6 

0.6 

Service provider 

School staff (%) 

96.4 

77.8 

77.8 

65.4 

65.4 

61.5 

Communities In Schools staff 

or partners (%) 

96.4 

88.9 

85.2 

92.3 

88.5 

80.8 

Other (%) 

35.7 

3.7 

22.2 

30.8 

23.1 

15.4 

Number of schools 

28 







SOURCE: Site coordinator survey (summer 2013). 

NOTES: The values presented include only those surveys in which the respondent answered the survey item; response rates 
vary among items. Items for the second and third measure were skipped if respondents reported "about 0%" of students faced 
the given issue at their school, or if services relating to the issue were not offered at the school. Missing values were excluded 
from calculations. The rates of missing responses to the above measures average to 7 percent, with a range of 0 percent to 19 
percent. 

a The percentages presented here are calculated from the mean of an 1 1-point answer scale, where respondents estimated the 
percentage of students in need at their school from 0 = “about 0%’’ to 10 = “about 100%.” 

b The values presented here are means on a three-point quality scale, where 0 = “Services don't address this issue, services are 
not appropriate for need,” 0.5 = “Services partially address this issue, more support is needed,” and 1 = “Services address this 
issue well.” 



sexual activity. About half of all case-managed students are considered to lack parental 
involvement or support, while nearly 40 percent experience family instability. 

Step 3: Case Planning 

Following the individual needs assessment conducted for each case-managed student, 
site coordinators create individual “case plans” that document each student’s areas of need, 
goals, and service plans for the school year — the third step depicted in Figure 3. 1 . In almost all 
the schools visited, site coordinators explained that they document individualized plans for the 
majority, if not all, of their students when beginning case management, and in most schools 
these plans are part of the official case management enrollment process. On the survey, site 
coordinators from all schools reported developing individualized case plans that include areas of 
need for every case-managed student. Almost all of them also indicated that case planning 
involves documenting goals for the year and outlining at least one specific service that students 
should receive as part of case management. Most site coordinators reported that students 
actively participate in the development of their case plans (as opposed to having a plan pre- 
scribed for them), with almost all site coordinators arranging multiple meetings with the 
students to create and agree on goals and benchmarks for measuring their progress. 

Although there appears to be consistency across the schools in documenting a case plan 
for students — including needs, goals, and services to be received — site coordinator and 
affiliate interviews suggest that there is variation in the levels of effort and detail involved. For 
instance, site coordinators and staff members from two affiliates described specific procedures 
for developing individual case plans for students, which include formulating “targeted issues” 
based on initial background assessments of student risk factors (in one affiliate) or actively 
engaging both students and parents to develop the plans in tandem with site coordinators. By 
contrast, site coordinators and staff members from another affiliate suggested that a case plan 
developed early in the year does not fully reflect the depth of a student’s needs and is more of a 
formality than a useful document to guide student services. Some site coordinators indicated a 
preference for tailoring services later in the year after they know their students better. Therefore, 
it seems that while all site coordinators develop case plans as expected with their case-managed 
students, not all rely on solidifying detailed plans that will guide the provision of services 
throughout the year. 

Step 4: Service Provision 

Based on their students’ needs assessments and subsequent case plans, Communities In 
Schools site coordinators work to provide and connect students with Level 2 services, as shown 
in the fourth step of Figure 3.1. Site coordinators reported on the provision of these services on 
surveys and also discussed these services during interviews. 
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Although site coordinators mentioned having assistants such as AmeriCorps members 
or college interns to whom they could delegate some service activities, the provision and 
coordination of these services were often conducted by site coordinators themselves. Site 
coordinators reported spending the greatest amount of their time on planning for or providing 
Level 2 services, as discussed in Chapter 2, and they stated during interviews that their respon- 
sibilities to their assigned case-managed students are a priority over their Level 1 duties. As 
shown in Table 3.1, at least 80 percent of site coordinators reported that Communities In 
Schools staff members or partners provide Level 2 services to address each of six categories of 
challenges faced by case-managed students. This table also indicates that site coordinators rely 
on the school’s staff to provide support to case-managed students in many areas, especially 
those related to poor academic performance, poor attendance, and behavior or discipline 
problems. For example, a site coordinator may steer a case-managed student to ongoing tutoring 
provided by teachers within the school and then help monitor that student’s participation in 
tutoring and academic performance. 

Furthermore, Table 3.1 indicates that while site coordinators believe they are able to 
serve a majority of case-managed students facing particular issues, they acknowledge that there 
is room for improvement in the services provided and that they are not able to meet some 
specific needs. For example, more than 90 percent of site coordinators reported being able to 
serve most or all students facing behavioral challenges and indicated that the services generally 
address the specified issue well. However, only about 60 percent of site coordinators reported 
being able to serve most or all students struggling with family instability, and most indicated 
that more support is needed. 

The site coordinator survey included detailed questions about service availability and 
frequency of service provision. As shown in Table 3.2, site coordinators reported a wide variety 
of Level 2 services. Site coordinators in almost all schools reported that the following services 
are available for case-managed students: academic assistance services, behavioral interventions, 
mentoring, links to basic needs and resources (for example, provision of food and school 
supplies), college planning and preparation, and family engagement activities. Of the services 
provided for case-managed students, academic assistance, mentoring, behavior interventions, 
and after-school programs are provided with the greatest frequency — at least once per week in 
a majority of the schools. 

In addition to providing infomiation about Level 2 services on the survey, site coordina- 
tors also discussed the details of case-managed service provision during interviews. In talking 
about the work they do with case-managed students, site coordinators commonly discussed 
three categories of services: academic, behavior and attendance, and social or life skills. It is 
important to note that neither site coordinators nor affiliate staff members considered these 
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Table 3.2 


Frequency of Case-Managed Service Provision 
As Reported by Site Coordinators 


Service 

Site Coordinators Reporting 
Service Is Offered (%) 

Average Y early 
Service Frequency 11 

Academic services 

Academic assistance 13 
Meeting with adult staff to discuss 

96.4 

30.2 

academic goals 

96.4 

18.7 

Behavior and attendance services 

Behavior intervention 11 

89.3 

26.3 

Anger management/ Conflict resolution 13 

85.7 

25.9 

Truancy prevention activities 

67.9 

20.0 

Social/life skills services 

Mentoring 13 

89.3 

31.3 

Gang intervention/prevention 

64.3 

16.2 

Pregnancy prevention 

64.3 

18.6 

Bullying prevention 

85.7 

13.1 

Drug abuse prevention 

67.9 

16.0 

Substance abuse support programs 1- 

53.6 

21.0 

Pregnancy/parent support programs 

50.0 

20.7 

Programs for adjudicated youth d 

39.3 

16.3 

Grief support programs 

57.1 

15.1 

Programs for LGBT youth" 

28.6 

6.0 

Basic needs/resources 

Linkages to basic needs/resources 

100.0 

16.2 

Physical health screening 

Exercise class or club/Obesity intervention 

50.0 

5.8 

and prevention 13 

42.9 

22.6 

College/career preparation 

College planning and preparation 

92.9 

15.8 

Career development/readiness programs 

85.7 

15.8 

Job shadowing or internship 

67.9 

7.3 

Enrichment/motivation services 

Community service/Service learning 

82.1 

8.3 

After- or before-school programs 13 ' 1 

64.3 

29.9 


(continued) 
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Table 3.2 (continued) 


Service 

Site Coordinators Reporting 
Service Is Offered (%) 

Average Y early 
Service Frequency 2 

Familv-related services 



Individual family engagement activities 

100.0 

7.4 

Parent group meetings and input activities 

71.4 

6.1 

School-sponsored activities for students 



and their families 

82.1 

8.5 

Number of schools 

28 



SOURCES: Site coordinator survey (summer 2013). 

NOTES: The percentages presented in the service frequency column include only those surveys in 
which the respondent reported that the service is offered at the school and answered the survey 
item related to service frequency. Response rates vary among services. Missing values were 
excluded from calculations. The rate of missing responses to the service frequency item average to 
5 percent, with a range of 0 percent to 27 percent. 

a The values presented here are approximations of the average number of times per year the 
service is offered. Times per year were calculated from respondents' answers to an original four- 
point frequency scale, where 1 = “once or twice a year” (1.5 times per year), 2 = “less than once a 
month” (4.5 times per year), 3 = “1-2 times a month” (13.5 times per year), and 4 = “one or more 
times a week” (36 times per year). 

b Of the site coordinators who reported that this service is offered, more than 50 percent 
reported that this service is offered one or more times per week. 
c The missing rate for this service frequency item is 27 percent. 

d The missing rate for this service frequency item is 18 percent. 

e The missing rate for this service frequency item is 25 percent. 

‘The missing rate for this service frequency item is 17 percent. 


service categories to be mutually exclusive. Because students often face many different types of 
challenges (for example, some site coordinators explained how poor attendance or poor aca- 
demic performance can be tied to struggles with problems at home or with peers), services may 
simultaneously address academic, behavioral, and socio-emotional needs. Thus the goal of 
connecting case-managed students with mentors might be to help their behavioral and personal 
development as well as to make them feel someone is holding them accountable for their school 
assignments. Specific types of services within the three categories most commonly discussed by 
site coordinators during interviews are described in more detail below. 

Academic Services 

Site coordinators’ descriptions of Level 2 academic services corroborated their survey 
responses, indicating that they were among the most frequently provided services to case- 
managed students. The types of academic services that site coordinators provide include 
homework assistance, study skill development, and working with teachers to address students’ 
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performance in class. While a high percentage of site coordinators reported that academic 
assistance is a key service they provide to case-managed students, only a minority of site 
coordinators from the visited schools described regularly reviewing course content or providing 
homework assistance directly to their case-managed students. Site coordinators in two affiliates 
specifically stressed their belief that it was important that they not be viewed by the school’s 
staff as academic tutors, teachers, or staff members who could be tasked by school administra- 
tors to perform academic instruction, such as filling in as substitute teachers. Instead, some site 
coordinators explained that they provide cocurricular services that allow a student to attend 
class regularly and focus on learning. As one affiliate representative noted, “I have mixed 
emotions about academic enhancement. I think we should be monitoring grades . . . helping 
when we have tutors. We are not teachers. We’re not the ones that should be an expert on how 
to get a kid to pass math class or physics.” Echoing this sentiment, many site coordinators 
explained that they view their academic support role to be about connecting their students to 
tutors, encouraging students to attend regular tutoring sessions held by their teachers, and/or 
working with teachers to understand where students are struggling in class. This may be the 
reason why site coordinators from almost all schools reported that school staff members provide 
services to address poor academic performance, as shown in Table 3.1. A few also described 
serving as a buffer between case-managed students and their teachers if and when conflicts 
arose regarding students’ in-class performance. 

Behavior and Attendance Sen/ices 

Another common area of service described by site coordinators and affiliate staff mem- 
bers included supporting case-managed students’ behavioral development. This category of 
Level 2 services includes such activities as offering guidance about improving conduct with 
peers and teachers, providing ongoing intervention when more serious behavioral or discipli- 
nary problems occur, and encouraging case-managed students to develop better habits for 
attending school and classes more consistently. Site coordinators from a large majority of 
schools said that they regularly address behavioral or disciplinary issues that their case-managed 
students experience during or out of class, statements supported by survey results presented in 
Table 3.1. 

Moreover, a few site coordinators also noted the importance of establishing relation- 
ships with their students to help them model positive behaviors, both in and out of school. As 
one coordinator said, “It’s a program that is helping our students to be successful in life ... to be 
productive citizens in society. We’re frying to instill in them something that will help them 
make the right decisions. We’re trying to plant seeds.” Site coordinators also mentioned the 
emphasis they place on modeling appropriate conduct and explaining its importance to students 
during their individual check-in sessions. There is variation by school and affiliate in the extent 
to which the site coordinators directly provide behavior-related services themselves and the 
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extent to which they connect students with existing services in their schools, with nearly 80 
percent of site coordinators reporting that they rely to some degree on the school staff to provide 
services that address attendance and behavior issues (as shown in Table 3.1). 

Social or Life Skills 

In addition to academic and behavioral services, Communities In Schools supports 
case-managed students’ social development. As displayed in Table 3.1, nearly three-quarters of 
site coordinators surveyed say they provide social or life skills services to most or all of their 
case-managed students who are engaged in high-risk social behavior. Site coordinators, affiliate 
staff members, and principals from many schools mentioned that a primary task site coordina- 
tors take on is to meet regularly with case-managed students in order to monitor their mental 
and emotional well-being. The goals of establishing one-on-one relationships between site 
coordinators (or their designees, such as AmeriCorps members, mentors, or interns) and 
students include offering continuous, individualized interactions and support, creating safe 
spaces for students to express themselves, and offering students new frames of reference about 
the world outside of school. The majority of this is done through regular check-in sessions. One 
site coordinator explained that part of her role in supporting students is about “following up 
with the student, making sure that the student is doing well, and letting them know that the 
support is there, the encouragement is there all the time.” The majority of site coordinators 
suggested during interviews that working through students’ individual personal and socio- 
emotional challenges involves making time for multiple one-to-one counseling sessions during 
the year. 

In addition, site coordinators discussed the importance of mentors and other volunteers 
in supporting case-managed students’ social development. Some affiliates have established 
partnerships with local churches and volunteer agencies that provide mentorship for students, 
while another affiliate has site coordinators connect case-managed students to service learning 
and volunteer opportunities around their local communities. Staff members from one affiliate 
explained that the community partnerships provide additional guidance and resources for site 
coordinators to reinforce positive behavior and life skills, including healthy living habits, and to 
teach students about domestic violence issues and prevention. 

Finally, site coordinators at most schools support students’ social and emotional devel- 
opment through the facilitation of weekly or biweekly topical discussion groups. While the 
large majority of site coordinators reported facilitating these regular groups, the topics or themes 
vary. For instance, site coordinators from one affiliate run two separate groups for case- 
managed students, separated by gender, using curricula with gender-specific discussion topics. 
In other affiliates, the case-managed groups are mixed gender. Some affiliates focus their 
groups on teaching and discussing “life skills” with students, which can include establishing 
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healthy relationships and social habits (such as hygiene and diet), while site coordinators from 
other affiliates focus their case-managed student groups on college preparation or career 
exploration activities. And some affiliates and schools provide multiple groups, spread across 
different parts of the school year, in order to cover a range of topics for their students. 

Step 5: Monitoring and Adjusting Services 

The final step of the case management process involves monitoring student progress 
and adjusting the services that students receive. As depicted in Figure 3.1, these two steps in the 
process are cyclical, as they may take place numerous times throughout the school year. 

Site coordinators in all schools reported on the survey that they can access student data 
and that they use it for tracking student progress during the year. Site coordinators in almost all 
schools visited told of such monitoring, primarily checking grades and attendance and writing 
progress reports about their case-managed students at least quarterly. Approximately three- 
quarters of the site coordinators reported being able to access student data in “real time,” while 
some site coordinators are able to access students’ records only at the end of each marking 
period. In addition to looking at student records, site coordinators at many of the schools 
discussed specifically monitoring progress against the goals they set with their case-managed 
students at the beginning of the year. 

While all site coordinators reported monitoring student progress during the school year, 
affiliates have developed different guidelines for this step. For example, one affiliate requires 
that site coordinators monitor students’ grades, attendance, and behavior referrals at least every 
six weeks and rate students’ progress in specific areas of need, using a numerical scale to 
indicate whether students are improving, declining, or staying the same. In another affiliate, site 
coordinators are required to report progress only once a semester for most case-managed 
students but more often for students with consistently low grades. In addition, that affiliate’s 
staff members explained that they monitor 10 percent of the case files to periodically check the 
sufficiency of service receipt and that their system flags students who are not getting services in 
the areas they are supposed to. In a third affiliate, site coordinators monitor student attendance 
on a weekly basis, regularly following up with students who are absent from school. 

In almost all schools visited, site coordinators discussed the need to monitor their stu- 
dents’ progress throughout the semester, and many mentioned making informal changes to 
the services students receive during the school year. Only a few site coordinators, however, 
explicitly stated that they make formal changes to students’ case plans based on student 
progress reviews or on observed changes in students’ needs. Moreover, few site coordinators 
were specific in their approaches to modifying goals or services listed on a student’s case 
plan. Staff members from many schools mentioned the expectations for site coordinators to 
revisit early goals to see how well students were progressing, but it was unclear whether and 
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how site coordinators were being held accountable for this task, or how often coordinators 
typically reexamined and adjusted initial case plans throughout the year. 


Summary and Discussion 

In general, interviews with Communities In Schools staff members suggest that the case 
management process is being implemented with fidelity across the 1 8 schools that were visited 
— that is, the site coordinators seem to be following each step of the case management process: 
identifying students in need of case management at their schools; assessing their needs; devel- 
oping case plans; working to help them stay on course academically and behaviorally, both 
through individual check-ins and through groups; and monitoring student progress. A closer 
look at how these steps are implemented, however, suggests that the approaches vary across 
affiliates. This variance especially exists in approaches to assessing student needs, to the 
development of case plans, and to the regularity of monitoring needs and adjusting case plans 
over time. Thus, while each step of the case management process appears to be implemented at 
all schools, affiliate-level guidelines are the likely reason for differences in the way site coordi- 
nators are implementing the steps. 

As discussed earlier, the Communities In Schools national office does not impose 
guidelines for the execution of the steps in the case management process, and thus each 
affiliate has flexibility to adjust the model to fit its local context. Considering the differences 
in school contexts, in terms of site coordinator backgrounds, principal leadership practices, 
availability of other support services, school culture, and other factors, it is easy to see why so 
much variation in implementing the program model may exist. Given these opportunities for 
variation, it is noteworthy that the steps in the case management process are in place across all 
the participating sites in the study. 

Nonetheless, while the variation is understandable, the data collected by the research 
team suggest some questions that Communities In Schools may want to consider regarding how 
case management is implemented across schools and affiliates — specifically, whether suggest- 
ing some standard practices in how needs are assessed and how case plans are used might lead 
to more consistency by site coordinators in the identification of student needs and thus to the 
prescription of services better targeted to those needs. Another consideration is the cyclical 
nature of the final steps of the case management process — monitoring student progress and 
adjusting services as needed. It is not clear how much service or case plan adjustment happens 
as a result of this monitoring. Given that one site coordinator indicated that it can be difficult to 
establish a satisfactory case plan at the start of the year, one might think this would be an 
important component of case management, but it is not certain whether this is so. This is one 
area that will be explored in greater detail in the next report from this evaluation. 
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Finally, one outstanding question regarding service provision relates to the quality of 
the services themselves. Given their many responsibilities, it is not clear whether site coordina- 
tors have the time to assess whether all the services being provided to case-managed students 
are of high quality. Further, site coordinators indicated on the survey that they are not always 
able to provide services to all or most students facing a given issue (especially if the need may 
fall beyond their typical scope of service provision, such as handling family instability issues). 
Their survey responses also indicated that more support is needed to fully address students’ 
needs. The provision of additional staff members or resources would probably help site coordi- 
nators in their service provision and monitoring tasks, but additional resources are often un- 
available, and additional staff members would require greater organization of their work. 
Another option in sites where site coordinators’ efforts may be spread too thin would be to 
focus even more intensively on a smaller group of students so that those who do end up on site 
coordinators’ caseloads receive more robust case management. In addition, a better understand- 
ing is required about how the responsibilities and actions of the site coordinator overlap with 
those of other staff members in the school (for example, counselors, social workers, and 
volunteers) and the extent to which other school staff members collaborate with site coordina- 
tors to provide case-managed services. The next report will further explore these questions 
across schools and affiliates. 

In Chapter 4, the focus of the report shifts from the provision or availability of services 
to students to the receipt of services by students. The chapter discusses the kinds of students 
recruited for case management and for the study and details the random assignment process. It 
then provides data on the services actually received by the case-managed students, critical for 
understanding what case-managed study students experience as part of the program, and 
examines differences in the services received by case-managed students and by students not 
receiving case management. If such differences are negligible, it is less likely that the two 
groups will diverge in outcomes such as attendance, behavior, and course performance. 
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Chapter 4 

Study Students and Support Services Received 


This chapter focuses on the students who participated in the study and the support services they 
received during the 2012-2013 school year. While the previous chapters discuss the schools in 
which Communities In Schools operates and provide details about each step of the case man- 
agement process, this chapter discusses the recruitment of students for case management and 
the study and the process used to randomly assign students to receive case management or not, 
which allows for a rigorous test of its impact. The chapter also examines the actual experiences 
of Communities In Schools Level 2 case-managed students and the students in the comparison 
group. It uses data from school records, student surveys, Communities In Schools management 
information systems, and interviews conducted during site visits to describe the students in the 
study, the level of case-managed services students received, and the differences between case- 
managed and non-case-managed students’ reports of in-school supports. In addition, it includes 
an investigation of whether the highest-risk students (that is, those with a recent history of 
course failures, suspensions, and very poor attendance) received different or more services 
compared with moderate-risk students. 

This infomiation is critical for understanding what case-managed study students experi- 
enced as part of the program. Further, investigating case-managed and non-case-managed 
students’ reports of service receipt helps highlight the difference between case management and 
“business as usual” for similar students in the study schools. If it does not appear that Commu- 
nities In Schools Level 2 case management provides students with support that goes above and 
beyond what students otherwise experience in the school, it would be less likely that differences 
would emerge between these two groups on the outcomes of interest. 

The highlights from this chapter include the following: 

• Random assignment was used to determine which students would receive 
Communities In Schools Level 2 case-managed services. Across the study 
schools, a total of 2,230 eligible students were included in the study, with 
even proportions of students assigned either to receive Level 2 case-managed 
services or not to receive case-managed services. Random assignment result- 
ed in two comparable groups of students with similar baseline characteristics. 

• Students in the Communities In Schools case-managed group received an 
average of 19.4 service contacts during the year, which amounted to an aver- 
age of 16.2 total Level 2 service hours. There was substantial variation in 
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service receipt among case-managed students, with some students receiving 
very low levels of services and others receiving high levels. 

• When compared with non-case-managed students, Communities In Schools 
case-managed students generally reported participating in support activities 
more frequently. Specifically, case-managed students were significantly 
more likely to report participating in individual and group meetings with 
adults in school, meeting with a mentor, participating in community ser- 
vice, and participating in positive behavior programs. Nonetheless, it is un- 
clear whether the magnitude of differences is great enough to affect student 
outcomes. 

• While there was a small amount of differentiation in the services Commu- 
nities In Schools provided to high- and moderate-risk students, high-risk 
case-managed students did not receive a greater number of hours or service 
contacts than moderate-risk students. If high-risk students, who are more 
likely to drop out of school, need more support than moderate-risk students, 
the provision of Level 2 services may need to be weighted more toward this 
group. 


Student Sample 

The schools included in this study serve students facing many challenges that may hinder their 
progress in school, but they also have numerous support services available (see Chapter 2 for 
additional details). Nonetheless, during site visit interviews and other visits before the study 
began, Communities In Schools and school staff members emphasized that a large proportion of 
the students in their schools were in need of the kind of supplementary support provided to 
Level 2 case-managed students. Given that Communities In Schools did not have the resources 
to support enough site coordinators to serve all the students in need in the study schools, 
students were assigned at random to either receive or not receive case management. This section 
includes details about the random assignment process, as well as descriptive information about 
the students in the study sample. 

Recruitment and Random Assignment 

After schools were selected for the study, the research team worked with Communities 
In Schools affiliates and site staff members to plan for student recruitment. Students are typical- 
ly identified for Communities In Schools case management through referrals and recommenda- 
tions from teachers, administrators, guidance counselors, or other student support staff members 
on campus. This identification process often delays the filling of caseloads until the second 
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semester. In order to enroll students in case management early in the school year, the research 
team assisted affiliates in working with their local school districts to collect data to preidentify 
students who would be eligible, based on whether the data indicated such issues as course 
failures, low grade point averages, poor attendance, or behavioral infractions. While using 
school records was standard practice for a few study schools, whose district provides Communi- 
ties In Schools with a list of at-risk students they may work with each year, it was new for most 
schools and affiliates in the study. Students were also considered to be eligible for case man- 
agement if they were identified through one of the more typical pathways — being referred to 
Communities In Schools by a school staff member or parent due to a recognized need (for 
example, academic, attendance, behavioral, or other personal or social difficulties) or identified 
directly by a Communities In Schools site coordinator (see Chapter 3 for additional details on 
student identification). 

As part of the student recruitment process, Communities In Schools affiliate and site 
staff members provided eligible students with information about Communities In Schools and 
the study. At each school, students who returned both the Communities In Schools and study 
consent forms were entered into MDRC’s random assignment database. 1 Students were includ- 
ed in the random assignment database regardless of whether their parent declined or consented 
to study participation, as service receipt was not contingent upon affirmative study consent. 
With the use of random assigmnent, every student eligible for case management (that is, 
students with consent to participate and demonstrated need for support) at each school had the 
same chance to be served. MDRC conducted random assignment lotteries for each school site to 
determine which students would fill the available Communities In Schools case management 
slots (the case-managed group) and which students would constitute a comparison group and 
continue with the standard services and supports otherwise available to them at their schools 
(the non-case-managed group). 2 

Although the recruitment and random assignment process was different from the typical 
recruitment process, involving additional effort at the start of the year, interviewed site coordi- 
nators generally reported that the students who were randomly assigned to their caseloads were 
similar to the nonstudy students on their caseloads and to students served in previous years. In 
five schools, site coordinators reported that a small number of students who were identified for 
case management using school records data performed at a higher level than they expected. For 
example, a student who failed a core course in the previous year might have been invited to 


1 These forms had to be signed by a parent or legal guardian unless the student was already an adult (18 
years old). 

‘In some schools, two lotteries were performed once a critical mass of students had been recruited so that 
the Communities In Schools staff could enroll students assigned to the case-managed group before reaching 
their school’s recruitment target. This occurred at 7 of the 28 schools in the study. 
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enter the lottery for case management based on the school record, but if the failure was an 
anomaly for that student, a site coordinator might find that he or she faced fewer challenges or 
had fewer needs than a typical case-managed student. On the other hand, Communities In 
Schools staff members also expressed a belief that using school records data to recruit students 
for the study helped identify some students for case management who might otherwise have 
fallen through the cracks. 

Baseline Student Characteristics 

The total study sample includes 2,230 students — 1,254 middle school students and 
976 high school students. This includes all students who were randomly assigned to the case- 
managed and non-case-managed groups who were enrolled at the time of random assignment 
and whose parents consented to their being part of the study. 3 The random assignment ratio was 
approximately 1:1, with 1,140 students assigned to the case-managed (program) group and 
1,090 assigned to the non-case-managed (comparison) group. 

This report discusses two separate samples of students who were included in subse- 
quent analyses. The school records analysis sample includes a total of 2,048 students, or 91.8 
percent of the total study sample. Students were excluded from this analysis sample if their 
course failure information was missing, course failure being one of the two primary outcomes 
for the study. In addition, students from the affiliate that contributed the smallest sample are 
excluded from this sample due to problems with acquiring school records data. The second 
sample is the student survey analysis sample, which includes a total of 2,093 students, or 93.9 
percent of the total study sample. Students were included in this analysis sample if they re- 
sponded to the spring 2013 student survey, regardless of whether school records were available 
for them. 4 

An examination of the student records analysis sample revealed that there were no sig- 
nificant differences between the case-managed and non-case-managed groups at baseline (that 
is, at the start of the study before the case-managed students began to receive Communities In 
Schools case-managed services) in students’ demographic characteristics or outcomes of 


3 A total of 2,297 students were randomly assigned to these groups, but 56 students’ parents did not con- 
sent to study participation. Also, district records show that 1 1 students exited their schools between turning in 
their consent form and random assignment, a fact not known to the research team until after random assign- 
ment was conducted. These 67 students have been excluded from the research sample. An additional 281 
students, an average of 10 per school, were randomly assigned to an ordered nonresearch waiting list group, 
which was established at each school to create a list of students who would receive services if additional case 
management slots became available. 

4 A total of 1,977 students, or 88.7 percent of the total study sample, are included in both analysis samples. 
See Appendix B for additional sample details. 
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Evaluation of Communities In Schools 
Table 4.1 

Baseline Characteristics of Students in the 
School Records Analysis Sample 


Baseline Characteristic (%) 

Case- Non-Case- 
Managed Managed Estimated 

Group Group Difference 

P-Value for 
Estimated 
Difference 

Race/ethnicity 




0.198 

Hispanic 

60.6 

59.7 

0.8 


Black, non-Hispanic 

35.1 

33.7 

1.4 


White, non-Hispanic 

1.8 

3.1 

-1.3 


Asian 

1.4 

2.0 

-0.7 


Other 

0.9 

1.3 

-0.4 


Male 

44.3 

44.5 

-0.2 

0.934 

Eligible for free/reduced-price lunch' 1 

53.9 

51.0 

2.8 ** 

0.045 

English as a second language 

12.0 

11.4 

0.6 

0.667 

Student qualifies for gifted program 

2.7 

3.6 

-0.8 

0.271 

Chronically absent 

9.4 

8.6 

0.8 

0.549 

Average attendance rate 

95.8 

95.8 

0.0 

0.989 

Failed at least one core course 

25.7 

26.2 

-0.5 

0.786 

Average core course marks 

80.1 

80.4 

-0.2 

0.493 

Joint test of difference between groups 0 (■£■ = 14.7) 



0.876 

Sample size c (N=2,048) 

1,029 

1,019 




(continued) 


interest, with the exception of students in the case-managed group being somewhat more likely 
to receive free or reduced-price lunch (see Table 4.1; see also Box 4. 1 for an explanation of how 
to interpret the baseline and impact tables in this report). The student sample is predominantly 
minority and low income, with approximately 60 percent Hispanic and 34 percent black 
students and more than 50 percent of students eligible for free or reduced-price lunch. 5 Approx- 
imately 55 percent of the students in the study sample are female, 12 percent are classified as 
students for whom English is a second language, and 3 percent were in gifted programs when 
the study began. Regarding this evaluation’s main outcomes of interest, the students in the study 
had an average attendance rate of nearly 96 percent, and approximately 9 percent were chroni- 
cally absent in the year before the study. Finally, students in the sample had average core course 


5 One school district was unable to provide students’ free/reduced-price lunch status. 
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Table 4.1 (continued) 


SOURCE: MDRC calculations based on student records obtained from school districts. 

NOTES: The analyses reported in this table are based on the school records analysis sample, 
which includes all students with course failure data for the 2012-2013 school year (one study 
district is excluded from this sample because student records data for this school year are not 
available). Due to small numbers, percentages for the Native American demographic group are 
not included in the table. 

The estimated differences between the case-managed group and the non-case-managed group 
are regression adjusted using ordinary least squares, controlling for random assignment blocks by 
school. The values in the column labeled “Case-Managed Group” are the observed means for 
students randomly assigned to the case-managed group. The “Non-Case-Managed Group” values 
are the regression-adjusted means for students randomly assigned to the non-case-managed group, 
using the observed distribution of the case-managed group across random assignment blocks as 
the basis for the adjustment. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between case -managed and non-case-managed 
groups. Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 
percent. 

a The missing rate for this characteristic is 27 percent. Data for this characteristic were not 
available for students in one of the six districts represented in the school records analysis sample. 

b A chi-square test was used to determine whether there is a systematic difference between the 
case-managed group and the non-case-managed group at baseline, based on the characteristics 
included in this table as well as indicators of missing data for all relevant student characteristics. 

c Due to missing values, the number of students included varies by characteristic. The average 
percentage of missing data on any given characteristic is 1 1 percent and ranges from 0 percent 
(gender) to 27 percent (free or reduced-price lunch eligibility). The percentage of missing data is 
high for free/reduced-price lunch eligibility because data for this characteristic were not available 
for students in one of the six districts in the school records analysis sample. 


marks of approximately 80 percent, and about one-quarter of the students failed at least one core 
course during the 2011-2012 school year. 

Comparisons of the case-managed and non-case-managed students in the student sur- 
vey analysis sample at baseline on family characteristics, educational aspirations, engagement 
with school, and adult support found the two groups to be similar, as presented in Table 4.2. 
This table shows that more than 50 percent of the students in the study lived with at least two 
parents or guardians, nearly 20 percent of students reported that their mother did not finish high 
school, and, among those with siblings of at least high school age, approximately one-third had 
at least one sibling who had dropped out of high school. Students in both the case-managed and 
non-case-managed groups reported feeling supported by adults in school, at home, and outside 
of school and home, 6 and they reported similar levels of engagement in school. These numbers 


6 The survey asked students to think about adults outside of their family and school. 


52 



Box 4.1 

Reading and Interpreting the Findings in the Report Tables 

Many of the tables in this report show the baseline characteristics of, services received by, or 
outcomes for students in the case-managed and non-case-managed groups — as well as the 
difference between these two groups. The values presented in these tables are derived as 
follows: 

“Estimated Impact” or “Estimated Difference.” This column shows the difference between 
students in the case-managed and non-case-managed groups with respect to their baseline 
characteristics, the services that they received, and their outcomes. The values in this column 
should be interpreted as the estimated difference or impact for the average student in the 
analysis sample. The statistical significance of the estimated difference or impact is indicated 
(*) when the p-value is less than or equal to 0.100, indicating no more than a 10 percent 
probability that the difference arose by chance. Estimated impacts are regression adjusted to 
account for random differences in the baseline characteristics of case-managed and non-case- 
managed students. All impact findings represent “intent-to-treat” estimates because some 
students in the case-managed group did not receive case management. See Appendix A for 
more information. 

“Case-Managed Group.” This column shows the observed baseline characteristics or out- 
comes of students randomly assigned to the case-managed group. Thus this column reflects the 
mean outcomes of the average student in the analysis sample. 

“Non-Case-Managed Group.” This column shows the counterfactual — that is, it provides an 
estimate of what the mean outcomes of case-managed students would have been had they not 
been randomly assigned to receive case management. The values in this column are regression 
adjusted based on the observed characteristics of students in the case-managed group. 

“Effect Size” (Chapter 5). This column shows the estimated impact scaled as an effect size. 
The effect size is a metric widely used for gauging whether the magnitude of a program’s 
impact is large or small. It is defined as the estimated effect of a program (or the difference in 
outcomes between case-managed and non-case-managed students) divided by the standard 
deviation of the outcome of interest. For example, an effect size of 0.20 represents an im- 
provement in student outcomes that is equal to 20 percent of the standard deviation of the 
student-level distribution for that particular outcome. The effect size, therefore, provides an 
indication of how much Communities In Schools improved a student’s outcomes relative to 
where they would have been in the outcome distribution for students in the program’s target 
population. In this report, effect sizes are calculated based on the standard deviation of the 
outcome of interest for students in the non-case-managed group, which reflects the expected 
variability in the outcome that one would find in the absence of Communities In Schools. 
Effect sizes of about 0.20 or less are generally considered “small,” effect sizes of about 0.50 
are considered “moderate,” and effect sizes of about 0.80 or more are considered “large.”* 

*Cohen (1988). 
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Evaluation of Communities In Schools 
Table 4.2 

Selected Baseline Characteristics of Students in the 
Student Survey Analysis Sample 



Case- 

Non-Case- 


P- Value for 


Managed 

Managed 

Estimated 

Estimated 

Item 

Group 

Group 

Difference 

Difference 

Household makeup 3 (%) 





Lives with 2 or more parents/guardians 

55.0 

52.1 

2.9 

0.190 

Lives with 1 parent/guardian 

38.2 

40.6 

-2.4 

0.276 

Lives with 1 or more grandparent(s) 

7.3 

9.0 

-1.7 

0.164 

Lives with his/her own child 

2.7 

2.4 

0.4 

0.608 

Language predominantly spoken at home b (%) 





English 

76.4 

76.1 

0.2 

0.894 

Not English 

23.6 

23.9 

-0.2 

0.894 

Parent educational attainment (%) 





Father 




0.329 

Not a high school graduate 

17.3 

20.9 

-3.7 


High school graduate or GED recipient 

25.9 

21.9 

4.0 


College graduate or higher 

12.2 

14.4 

-2.2 


Don't know 
Mother 

44.6 

42.7 

1.9 

0.852 

Not a high school graduate 

18.3 

21.2 

-2.9 


High school graduate or GED recipient 

28.4 

27.6 

0.8 


College graduate or higher 

23.7 

24.2 

-0.5 


Don't know 

29.6 

27.0 

2.7 


Did any siblings leave high school before 
graduation? 0 (%) 





None left high school 

64.8 

66.1 

-1.4 

0.590 

At least 1 left high school 

35.2 

33.9 

1.4 

0.590 

Student engagement with school (l-4) d 

2.85 

2.87 

-0.01 

0.656 

How far would you like to go in school with your 
education? (%) 




0.935 

Some high school 

0.7 

1.2 

-0.5 


Finish high school 

11.7 

9.1 

2.6 


Some college or trade/technical school 

6.0 

5.5 

0.5 


Finish college or trade/technical school 

45.1 

44.9 

0.1 


Graduate school after college 

29.9 

32.2 

-2.3 


Don't know 

6.6 

7.2 

-0.6 


(continued) 
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Table 4.2 (continued) 


Item 

Case- 

Managed 

Group 

Non-Case- 

Managed 

Group 

Estimated 

Difference 

P- Value for 
Estimated 
Difference 

How far do you think you will actually go in school or 
with your education? (%) 

Some high school 

1.6 

1.0 

0.6 

0.937 

Finish high school 

12.4 

12.5 

-0.1 


Some college or trade/technical school 

11.0 

10.6 

0.3 


Finish college or trade/technical school 

38.3 

41.0 

-2.7 


Graduate school after college 

25.5 

26.1 

-0.6 


Don't know 

11.3 

8.8 

2.4 


Caring adult at home ( 1 -4 ) e 

3.38 

3.40 

-0.03 

0.236 

£ 

Caring adult at school (1-4) 

3.31 

3.28 

0.03 

0.333 

Caring adult outside of home or school (1-4) 8 

3.46 

3.49 

-0.03 

0.367 


Joint test of difference between groups 



0.966 

Sample size 11 (N = 2,093) 

1,058 

1,035 



SOURCE: MDRC calculations based on fall 2012 baseline student survey. 

NOTES: The analyses reported in this table are based on the student survey analysis sample, which 
includes all students who responded to the spring 2013 follow-up student survey. 

The estimated differences between the case-managed group and the non-case-managed group are 
regression adjusted using ordinary least squares, controlling for random assignment blocks by school. 
The values in the column labeled “Case-Managed Group” are the observed means for students randomly 
assigned to the case-managed group. The “Non-Case-Managed Group” values are the regression- 
adjusted means for students randomly assigned to the non-case-managed group, using the observed 
distribution of the case-managed group across random assignment blocks as the basis for the 
adjustment. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between the case -managed and non-case-managed 
groups. Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

“These survey categories are not mutually exclusive. 

b Of those who responded that English was not the predominant language at home, 92 percent 
reported speaking Spanish at home. 

Respondents without siblings old enough for high school are omitted. 

d Scale based on responses to survey questions 13a-13e, ranging from 1 = “not at all true” to 4 = 

“very much true.” Cronbach's alpha = 0.83. 

e Scale based on response to survey questions 6a-6g, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach's alpha = 0.81. 

f Scale based on response to survey questions 12a-12f, ranging from 1 = "not at all true” to 4 = “very 
much true.” Cronbach's alpha = 0.89. 

=Scale based on response to survey questions 10a- lOf, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach's alpha = 0.89. 

h Due to missing values, the number of students included varies by characteristic. The sample size 
reported here is for the student survey analysis sample. The percentage of missing data on any given 
characteristic averages to 8 percent and ranges from 3 percent to 35 percent. 
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suggest that, as expected, random assignment created two groups that were comparable at the 
start of the 2012-2013 school year. 


Case-Managed Service Participation 

This section provides details about the services received by Communities In Schools case- 
managed students by examining the type, frequency, and amount of services provided to case- 
managed students (see the “Activities” column in Figure 1.1, the Communities In Schools case 
management logic model). The research team learned about the amount of Level 2 services 
students received from interviews with site coordinators and from management information 
system (MIS) data documented by Communities In Schools site coordinators. 7 Communities In 
Schools staff members explained that whenever a case-managed student receives any type of 
Level 2 service, that service should be documented in the system. Site coordinators and affiliate 
staff members reported that, in most cases, case-managed students’ participation in Level 1 
services is not recorded in the system. The MIS data provide information about the services 
students received in up to eight categories — academic, behavior, attendance, social or life 
skills, basic needs and resources, college and career preparation, enrichment or motivation, and 
family-related services (see Box 4.2 for additional details). For every case-managed student, site 
coordinators track both service contacts — each time any service is received — and the length 
of time each service contact lasts. 

Overall Service Contacts and Service Hours 

After being randomly assigned to the case-managed group, students were enrolled in 
case management for just under 30 calendar weeks, on average, or about three-quarters of the 
2012-2013 school year. As Table 4.3 indicates, 89 percent of students in the survey analysis 
sample who were assigned to the Communities In Schools case-managed group had at least 1 
documented direct service contact during the school year. 8 They received an average of 19.4 
service contacts during the year and an average of 16.2 service hours. For the portion of the 


7 The research team received MIS data for students in the case-managed group. The data varied by affdiate; 
the Communities In Schools national office provided information for the three affiliates using the national MIS 
database, and two affiliates provided MIS data from their own or state systems. In order to compare service 
provision information across the three different data systems, the research team collapsed the MIS data into the 
eight service types described above. However, not all affiliates had information regarding all eight categories. 
This information represents the best estimate of service receipt from these data, but due to the inconsistencies 
in the way information is gathered by the three systems, there may be some suppression or inflation of how 
much service provision students in the sample received. 

8 The remaining 1 1 percent of the surveyed students assigned to the case-managed group did not have any 
direct services documented. 
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Box 4.2 

What Kinds of Activities Are Included in Each Service Type? 

Academic services. Adult or peer tutoring, homework assistance, study skills activities, 
student-teacher conferences 

Behavior services. Conflict resolution groups, anger management or other behavioral counsel- 
ing, violence prevention activities, behavior monitoring and interventions 

Attendance services. In-person attendance check-ins and planning 

Social or life skills services. Goal-setting activities; self-esteem enhancement activities; girls’ 
or boys’ groups; social, relationship, and communication activities; team-building games and 
activities; crisis and grief counseling services 

Basic needs and resources. Provision of school supplies; assistance with utilities, rent, etc.; 
food and clothing assistance; health activities and checkups 

College and career preparation. College admissions preparation and assistance, career 
counseling, college visits and career field trips, college awareness activities and programs 

Enrichment or motivation services. Community service, field trips unrelated to college or 
career preparation, sports or exercise activities, scouting, arts and crafts, student recognition 
activities and incentives 

Family-related services. Parent education, home visits, parent conferences and contacts, 
parent and family events and activities, family counseling 


school year during which students were enrolled in case management, this amounts to just over 
2.5 service contacts per month, each lasting about 50 minutes, on average. 9 

In addition to exploring the MIS data to understand the average number of service con- 
tacts and hours received by Communities In Schools case-managed students, the research team 
examined variation in service receipt. As shown in Figure 4.1, the relationship between overall 
service contacts and hours was linear for most of the students in the sample such that students’ 
service hours increased by roughly 1 hour for every 2 additional service contacts they received. 
Within this group, some case-managed students appear to have experienced relatively long 
contacts, while others appear to have experienced shorter check-ins, as each service contact 
lasted for an average of 1 hour for some students but an average of only 15 minutes for others. 


"’Although students were not enrolled in case management for the entire school year, this averages out to 
just under 2 service contacts, lasting between 45 and 50 minutes each, per month in a 10-month school year. 
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Evaluation of Communities In Schools 


Table 4.3 


Service Receipt for Case-Managed Students 
in the Student Survey Analysis Sample 



Percentage 

Average 

Average 


of Students 

Number of Times 

Total Hours of 

Services Offered 

Receiving Service 

Service Received 2 

Service Received 

Overall service receipt 

Any service type 

88.9 

NA 

NA 

Across all service types 

NA 

19.4 

16.2 

Service receipt bv service type 

Academic 

76.6 

4.1 

4.3 

Behavior 

50.1 

4.9 

4.8 

Social/life skills 

60.5 

7.4 

5.8 

Basic needs/resources 

56.0 

2.0 

1.6 

College/career preparation 

42.6 

2.0 

3.4 

Enrichment/motivation 

52.4 

1.8 

4.8 

Family-related 

47.7 

1.2 

0.4 

Attendance 

15.7 

2.4 

0.9 

Number of students' 3 (N=l,058) 


SOURCES: Data from Communities In Schools Data Management system (CISDM) and two local 
CIS service provision databases (2012-2013). 

NOTES: The analyses above are for case-managed (program group) students only, and are based on 
the student survey analysis sample, which includes all students who responded to the spring 2013 
follow-up student survey. Outliers and students with missing data are excluded from the analyses; 
for details, see supplementary Appendix D in Corrin et al. (2015), available at www.mdrc.org. 

The services offered are not mutually exclusive; a student could have received more than one 
type of service over his or her enrolled period. 

Calculations for the percentage of students receiving a given service are based on a consistent 
denominator of 1,058 case-managed students. However, not all service types were available in each 
data source. All data sources provided academic, behavior, and social/life skills service types. The 
basic needs/resources, college/career preparation, enrichment/motivation, and family-related 
service types were available for 84.3 percent of student records. The attendance service type was 
available for 58.1 percent of student records. 

Rounding may cause slight discrepancies in averages and percentages. 

a The services are received over the entire time a student was enrolled in CIS case management 
during the 2012-2013 school year. The average number of days enrolled for case-managed students 
is 207. The averages presented for number of times and total hours of service include only those 
students who received the associated service. 

b Of all case-managed group students (N=l,058), 1 1.2 percent did not receive direct case 
management services. In addition, 2 percent of all non-case-managed group students (N= 1,090) 
received case management services but are not included in this table. 
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Evaluation of Communities In Schools 


Figure 4.1 

Service Hours by Service Contacts for Case-Managed Students 

80 

70 



0 10 20 30 40 50 60 70 80 

Total Service Contacts 


SOURCES: Data from Communities In Schools Data Management system (CISDM) and two 
local CIS service provision databases (2012-2013). 

NOTES: The data reported in this figure represent case-managed (program group) students in 
the student survey analysis sample, which includes all students who responded to the spring 
2013 follow-up student survey. Outliers and students with missing data are excluded from the 
figure; see supplementary Appendix D in Corrin et al. (2015) for details. 


Despite this variation, for most case-managed students, more hours of service were associated 
with more service contacts. 

But Figure 4. 1 also shows that there was a group of students with a small number of 
service contacts but a disproportionately large number of hours — represented by the spike in 
hours on the scatterplot between 1 and 5 service contacts. Constituting approximately 18 
percent of the case-managed student sample, these students had 5 or fewer service contacts and 
more than 10 service hours, with many students averaging upward of 5 hours per service 
contact. In formation shared by Communities In Schools’ local and national staff members 
suggests that these students probably attended half- or full-day activities a few times during the 
year — most likely field trips. For example, Communities In Schools case-managed students 
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often have opportunities to attend trips to colleges or spend a day doing community service with 
their site coordinator and other students. 

During interviews, Communities In Schools site coordinators and affiliate staff mem- 
bers explained that the variation in the services students receive is based on students’ individual 
circumstances, needs, and personalities. Site coordinators from half of the schools visited 
described certain splits in their caseloads, explaining that some “lighter touch” students may 
need only some academic tutoring or attendance check-ins, whereas other students have 
additional needs, require more attention, and/or experience more intense life circumstances that 
impede their progress at school, and these students may receive more hours or services. But 
some site coordinators also reported that students’ service contacts and hours may be based on 
their openness to receiving services — to sharing their problems and seeking out site coordina- 
tors to discuss particular needs or personal challenges. Such students may receive more services 
than others who are more reserved or harder to reach. 

Service Receipt by Service Type 

In addition to providing information about the overall number of service contacts and 
service hours for case-managed students, Table 4.3 includes information about the different 
types of services students received. The services received by the greatest proportion of case- 
managed students were academically focused. More than 75 percent of case-managed students 
received academic services, with those students receiving an average of 4 service contacts 
totaling 4.3 hours during the year. Additionally, 61 percent of case-managed students received 
social or life skills services, with averages of 7.4 service contacts and 5.8 hours, and half 
received a behavior-related service, averaging nearly 5 service contacts and 5 hours of service. 
This information aligns with site coordinators’ statements during interviews that students most 
frequently received academic and behavioral services. Nearly all site coordinators discussed 
facilitating social skill- and behavior-focused groups with case-managed students, and most 
discussed implementing behavioral interventions and modifications, as described in Chapter 3. 

More than half of all case-managed students also received services focused on enrich- 
ment and motivation, averaging under 2 service contacts but nearly 5 hours of service per 
student. Many site coordinators said that field trips are an important aspect of enrichment 
services for case-managed students, which may explain why the service hours are high relative 
to some of the other service types even though service contacts are low. The smallest proportion 
of students — approximately one in six — received attendance services, and those who did 
received less than 1 hour of service. However, site coordinators and affiliate staff members 
explained during interviews that the attendance-related work done by most site coordinators 
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involves monitoring students’ attendance records rather than providing direct services, so such 
work would not be documented in the system and linked to individual students. 10 

Such monitoring is one way students may receive support from site coordinators that 
does not directly (physically) involve them. Although most of the affiliates in the study do not 
track these indirect support services as part of students’ individual records, one affiliate does use 
a system that allows site coordinators to log indirect actions related to case-managed students. 
In that affiliate, all case-managed students received indirect services. Specifically, site coordina- 
tors recorded an average of 8 instances of indirect service totaling an average of approximately 
2 hours per student. Although the time that site coordinators spend on these tasks may vary by 
affiliate, this supports the notion that, in addition to getting connected to and provided with the 
direct support services described above, one component of receiving case management is that 
students have an adult in the school who spends time checking on them during the school year. 


Services Reported by Case-Managed and 
Non-Case-Managed Students 

As discussed in Chapter 2, the schools in which Communities In Schools operates appear to 
have many services available to students that aim to address the diverse issues they may face. 
The students in the non-case-managed group presumably have access to the wide variety of 
services offered in their schools, including services provided or brokered by Communities In 
Schools as Level 1 services. While MIS data on the services received by non-case-managed 
students are not collected across the study schools, the research team administered student 
surveys asking case-managed and non-case-managed students about the activities they partici- 
pated in and the services they received. Using this common method for both groups of students 
provides comparable data across all students in the study so that differences between the two 
groups can be clearly assessed. Given that these two groups of students were equivalent at the 
time of random assigmnent, this comparison indicates how service receipt, as reported by 
students, differs as a result of Communities In Schools case management. 

Table 4.4 details the services reported by case-managed and non-case-managed stu- 
dents. For a majority of the support activities included on the survey, case-managed students 
reported higher levels of participation than non-case-managed students, and a number of the 
differences between the two groups were statistically significant. Specifically, case-managed 


10 As indicated in Table 4.3, data on student service receipt came from three different service provision 
databases. Not all service types were available in each data source. The basic needs/resources, college/career 
preparation, enrichment/motivation, and family-related service types were available for 84.3 percent of student 
records. The attendance service type was available for 58. 1 percent of student records. 
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Evaluation of Communities In Schools 


Table 4.4 


Impacts on Student Survey Outcomes for the 
Student Survey Analysis Sample 


Item 

Case- 

Managed 

Group 

Non-Case- 
Managed Estimated 
Group Difference 

P- Value for 
Estimated 
Impact 

In-school support activities (times per year) a 
Received tutoring or homework help 

6.8 

6.9 

0.0 

0.966 

Met with a mentor 

3.5 

2.2 

1.3 *** 

0.000 

Community service/Volunteering 

2.8 

2.3 

0.5 * 

0.060 

Exercise class or club 

2.4 

2.4 

0.0 

0.910 

Positive behavior program 
like drug-free/antibullying 

2.8 

2.2 

0.6 ** 

0.031 

College planning activity 

5.3 

4.8 

0.5 

0.190 

Career planning activity 

3.2 

2.9 

0.3 

0.403 

Job shadowing/internship 

1.6 

1.7 

-0.1 

0.620 

After-school program 

4.8 

5.6 

-0 .9 ** 

0.031 

Received assistance like school supplies, 
food, bus pass, clothing, or gifts 

2.8 

2.7 

0.1 

0.736 

Health checkup 

3.2 

3.6 

-0.3 

0.270 

In-school meetings with adults (times per year) a 
Individual meeting about academics 

7.1 

6.2 

0.9 ** 

0.023 

Individual meeting for support during a 
life-changing event 

2.9 

2.6 

0.3 

0.311 

Individual meeting about personal 
goals and behavior 

5.9 

4.7 

1,2 *** 

0.003 

Group meeting about academics 

5.5 

4.4 

*** 

0.003 

Group meeting for support during 
a life-changing event 

2.7 

2.4 

0.3 

0.273 

Group meeting about personal 
goals and behavior 

4.7 

3.7 

1.0 *** 

0.009 

Group meeting for social activities 

5.9 

2.9 

2,9 *** 

0.000 


(continued) 


students reported engaging in the following activities significantly more often than non-case- 
managed students: meeting with a mentor, participating in community service or volunteering, 
and participating in a positive behavior program. Case-managed students reported participating 
in after-school programs less often than non-case-managed students, but it is possible that they 
do not view some Communities In Schools activities that take place after school hours, such as 
meeting with a mentor or being in an exercise club, as after-school programs. 
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Table 4.4 (continued) 



Case- 

Non-Case- 


P- Value for 


Managed 

Managed 

Estimated 

Estimated 

Item 

Group 

Group Difference 

Impact 

Met with an adult at school to set 
specific goals for the year (%) 
Earned rewards for improving grades, 

64.4 

52.6 

1 19 *** 

0.000 

attendance, or behavior, or for reaching 
or making progress toward goals (%) 

58.7 

55.9 

2.8 

0.192 

An adult in school connected student to support 





programs or help outside of school (%) 

50.6 

42.3 

8.3 *** 

0.000 

Number of students’ 5 (N=2,093) 

1,058 

1,035 




SOURCE: MDRC calculations based on the spring 2013 follow-up student survey. 

NOTES: The analyses reported in this table are based on the student survey analysis sample, which 
includes all students who responded to the spring 2013 follow-up student survey. 

The estimated differences between the case-managed group and the non-case-managed group are 
regression adjusted, controlling for random assignment blocks by school, as well as the following 
baseline characteristics: race, gender, free or reduced-price lunch status, English as a second language, 
and whether qualified for a gifted program. 

The values in the column labeled “Case-Managed Group” are the observed means for students 
randomly assigned to the case-managed group. The “Non-Case-Managed Group” values are the 
regression-adjusted means for students randomly assigned to the non-case-managed group, using the 
observed mean covariate values for the case-managed group as the basis for the adjustment. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between the case-managed and non-case -managed 
groups. Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

a The times per year measure was created by combining a measure of duration (“For how long did you 
do this activity?”) and a measure of frequency (“Flow often did you do this activity?”) for each of the 
support activities above in the spring 2013 follow-up student survey. The duration measure was 
converted from a four-point scale to an estimate of the number of weeks per school year a student 
participated in the activity: “1 never did this activity” = 0 weeks, “less than half of the school year” = 9 
weeks, “about half of the school year” = 18 weeks, and “most or all of the school year” = 27 weeks. The 
frequency measure was converted from a four-point scale to an estimate of the number of times per 
week a student participated in the given activity: “I never did this activity” = 0, “less than once a month” 
= 0.125, “1-2 times a month” = 0.375, and “one or more times a week” = 1. 

b The sample size reported in the table is for the full student survey analysis sample. However, the 
sample size varies across outcomes due to missing data. The percentage of missing data on any given 
outcome does not exceed 5 percent. 


Table 4.4 also includes information about students’ reports of in-school meetings with 
adults. Case-managed students reported participating in individual and group meetings led by 
adults focused on their academics or personal goals and behavior significantly more often than 
non-case-managed students. They also reported participating in group meetings for social 
activities more often than non-case-managed students. Finally, students in the case-managed 
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group were more likely than non-case-managed students to report meeting with an adult at 
school to set specific goals for the year and to report that an adult in school connected them with 
support programs or help. 

These results support the infonnation shared by site coordinators that is discussed in 
Chapter 3. A majority of site coordinators told of meeting with each case-managed student to 
discuss goals and academics, conducting group sessions with case-managed students, connect- 
ing students with mentors, and engaging students in community service activities. However, the 
absolute levels reported by both case-managed and non-case-managed students for many 
activities were not high. For example, case-managed students reported meeting with a mentor 
an average of 3.5 times per year, compared with 2.2 times per year for non-case-managed 
students, and non-case-managed students reported participating in an after-school program 5.6 
times per year, compared with 4.8 times for case-managed students. Nonetheless, it is important 
to keep in mind that these numbers represent averages — for students who are not facing major 
struggles, these levels of support may be more than enough, while they may not be sufficient for 
students who are struggling a great deal. Further, within both the case-managed and non-case- 
managed groups, there are students who received very few services and students who received 
many services. One key question, then, is whether the students who have the greatest needs 
received the greatest levels of support. This question is explored in the section below. 


Student Service Receipt by Risk Category 

While the primary analyses for this study examine all study students, both Communities In 
Schools site coordinators and school staff members noted that the types of students eligible for 
case management demonstrate varied levels of need. Communities In Schools staff members 
explained that certain students have more intensive needs, evidenced by failing grades, very 
poor attendance, behavioral infractions, and other social and personal challenges, and should 
therefore receive more services than other case-managed students. With this in mind, the 
research team conducted a set of analyses to examine variation in service receipt for students 
who could be considered “high risk” compared with the “moderate-risk” students in the sample 
to determine whether the students who appear to have the greatest needs receive the greatest 
levels of service. 11 Using school records data, the research team classified students as being high 
risk if they were either chronically absent (less than 90 percent attendance), failed one or more 
core courses, or were suspended in the 2011-2012 school year — the year before the study. 
Approximately 40 percent of the students in the Communities In Schools case-managed group 


The two groups are referred to as high and moderate risk rather than high and low risk because being 
eligible for case management itself implies some level of risk. 
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fall into the high-risk group. The other 60 percent of case-managed students were classified as 
moderate risk. 12 

Table 4.5 details Level 2 service receipt for high- and moderate-risk case-managed stu- 
dents. Overall, the primary finding from this investigation is that the services received by higli- 
and moderate-risk students were quite similar, with no significant differences in their overall 
number of service contacts or hours. There were, however, minor differences between these 
groups that might suggest some distinction between high- and moderate-risk students. The MIS 
data indicate that the percentage of case-managed students in the high-risk group who did not 
receive any services was smaller — 8.7 percent, compared with 12.8 percent for moderate-risk 
students. It may be that high-risk students are more difficult to reach (for example, they may 
come to school less frequently and have disciplinary infractions), so these numbers suggest 
additional effort on the part of site coordinators to connect with these students. There was also 
some differentiation in the types of services students received, as high-risk students were more 
likely to receive services focused on behavior and social or life skills, but they were somewhat 
less likely to receive college and career preparation services than moderate-risk students. 

Moderate-risk students received more hours of college or career preparation, the only 
service type for which there was a statistically significant difference between the two risk 
groups in hours of service receipt. There were only a few relatively minor differences in service 
contacts, most notably in academic services, where the high-risk students received support more 
often than the moderate-risk students. 

Table 4.6 shows differences in reported service receipt between the case-managed and 
non-case-managed students broken down by risk category, adding more detail to the findings 
discussed earlier, with reference to Table 4.4. These findings fall into four categories based on 
whether there were overall differences between case-managed and non-case-managed students 
(Table 4.4) and whether there were differences between those two groups for the high-risk or 
moderate-risk students (Table 4.6). 

First, Table 4.6 shows a significant difference between case-managed and non-case- 
managed students in both the high- and moderate-risk groups in their reports of participating in 
particular support activities. These activities include mentoring; adult-facilitated group meetings 
related to academics, to personal goals and behavior, and to social activities; goal setting with 
adults; and getting connected to out-of-school supports — activities where there was also an 
overall difference between case-managed and non-case-managed students, as shown in Table 


12 Percentages are calculated based on the school records analysis sample, which includes 1,029 case- 
managed students. 
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Evaluation of Communities In Schools 


Table 4.5 


Service Receipt for Case-Managed Students in the 
Student Survey Analysis Sample, by High- and Moderate-Risk Students 



Percentage of Students 
Receiving Service 

Average Number of Times 
Service Received 2 

Average Total Hours 
of Service Received 

Services Offered 

High-Risk 

Students ’ 3 

Moderate-Risk 

Students 

High-Risk 

Students 

Moderate-Risk 

Students 

High-Risk 

Students 

Moderate -Risk 
Students 

Overall service receipt 

Any service type 

91.3 

87.2 ** 

NA 

NA 

NA 

NA 

Across all service types 

NA 

NA 

19.3 

19.4 

16.1 

16.3 

Service receipt bv service type 

Academic 

76.5 

76.6 

4.6 

3 g *** 

4.5 

4.2 

Behavior 

56.1 

46.0 *** 

4.8 

5.0 

4.7 

4.9 

Social/life skills 

65.3 

57 5 *** 

7.2 

7.6 

6.0 

5.7 

Basic needs/resources 

53.5 

57.6 

1.9 

2.1 

1.6 

1.7 

College/career preparation 

39.2 

44.9 * 

2.0 

2.0 

3.0 

3.7 * 

Enrichment/motivation 

52.1 

52.5 

1.6 

1.8 * 

4.8 

4.8 

Family-related 

46.7 

48.4 

1.2 

1.1 * 

0.4 

0.4 

Attendance 

17.4 

14.6 

2.5 

2.3 

1.0 

0.8 

Number of students 0 

(N=426) 

(N=632) 






(continued) 



Table 4.5 (continued) 

SOURCES: Data from Communities In Schools Data Management system (C1SDM) and two local CIS service provision databases (2012- 
2013). 

NOTES: The analyses above are for case-managed (program group) students only, and are based on the student survey analysis sample, 
which includes all students who responded to the spring 2013 follow-up student survey. Outliers and students with missing data are 
excluded from the analyses; for details, see supplementary Appendix D in Corrin et al. (2015), available at www.mdrc.org. 

The services offered are not mutually exclusive; a student could have received more than one type of service over his or her enrolled 
period. 

Calculations for the percentage of students receiving a given service are based on a consistent denominator of 426 high-risk students and 
632 moderate -risk students. However, not all service types were available in each data source. All data sources provided academic, 
behavior, and social/life skills service types. The basic needs/resources, college/career preparation, enrichment/motivation, and family- 
related service types were available for 84.3 percent of student records. The attendance service type was available for 58.1 percent of 
student records. 

Rounding may cause slight discrepancies in averages and percentages. 

A t-test was conducted to test for differences between findings for high- and moderate -risk students. Statistical significance levels are 
indicated as: *** = 1 percent, ** = 5 percent, * = 10 percent. 

a The services were received over the entire time a student was enrolled in CIS case management during the 2012-2013 school year. The 
average number of days enrolled for case-managed students is 207. The averages presented for number of times and total hours of service 
include only those students who received the associated service. 

b High-risk students are defined as those who were chronically absent, who failed a core course, or who were ever suspended in the 2011- 
2012 school year. Moderate -risk students include those who were never chronically absent, never failed a core course, and were never 
suspended in the 2011-2012 school year. 

c Of all case-managed group students (N=l,058), 1 1.2 percent did not receive direct case management services. In addition, 2 percent of 
all non-case-managed group students (N=l,090) received case management services but are not included in this table. 
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Table 4.6 


Impacts on Student Survey Outcomes for High- and Moderate-Risk Students 
in the Student Survey Analysis Sample 


Item 

Case- 

Managed 

Group 

Non-Case- 
Managed Estimated 
Group Difference 

P- Value for 
Estimated 
Impact 

In-school support activities (times per year) 3 
Received tutoring or homework help 

High-risk students 

7.1 

7.4 

-0.3 

0.668 

Moderate-risk students 

6.5 

6.6 

-0.1 

0.811 

Met with a mentor 

High-risk students 

3.4 

2.2 

\ 2 *** 

0.010 

Moderate-risk students 

3.2 

1.9 

1.3 *** 

0.001 

Community service/Volunteering 

High-risk students 

2.8 

1.9 

0.9 ** 

0.038 

Moderate-risk students 

2.5 

2.3 

0.2 

0.575 

Exercise class or club 

High-risk students 

2.2 

2.3 

-0.1 

0.769 

Moderate-risk students 

2.5 

2.3 

0.2 

0.695 

Positive behavior program like drug-free/antibullying 

High-risk students 3.2 

2.5 

0.7 

0.146 

Moderate-risk students 

2.6 

2.2 

0.4 

0.297 

College planning activity 

High-risk students 

5.5 

4.4 

1.1 * 

0.095 

Moderate-risk students 

5.1 

4.9 

0.3 

0.619 

Career planning activity 

High-risk students 

3.5 

3.1 

0.4 

0.418 

Moderate-risk students 

2.9 

3.0 

-0.1 

0.874 

Job shadowing/internship 

High-risk students 

1.9 

1.8 

0.1 

0.781 

Moderate-risk students 

1.3 

1.7 

-0.4 

0.207 

After-school program 

High-risk students 

4.1 

5.9 

_pg *** 

0.009 

Moderate-risk students 

5.1 

5.5 

-0.5 

0.402 

Received assistance like school supplies, 
food, bus pass, clothing, or gifts 

High-risk students 

2.8 

3.3 

-0.4 

0.407 

Moderate-risk students 

2.9 

2.4 

0.5 

0.178 

Health check-up 

High-risk students 

3.4 

4.1 

-0.6 

0.224 

Moderate-risk students 

3.2 

3.4 

-0.3 

0.521 


(continued) 
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Table 4.6 (continued) 



Case- 

Non-Case- 


P- Value for 


Managed 

Managed 

Estimated 

Estimated 

Item 

Group 

Group Difference 

Impact 

In-school meetings with adults (times per year) 





Individual meeting about academics 





High-risk students 

7.7 

6.6 

1.1 

0.134 

Moderate-risk students 
Individual meeting for support during a 

6.8 

6.0 

0.9 

0.109 

life-changing event b 
High-risk students 

3.9 

2.8 

1.1 ** 

0.045 

Moderate-risk students 

2.3 

2.5 

-0.2 

0.611 

Individual meeting about personal goals and behavior 




High-risk students 

6.3 

5.3 

1.0 

0.154 

Moderate-risk students 
Group meeting about academics 

5.9 

4.4 

1.5 *** 

0.006 

High-risk students 

5.7 

4.2 

1.5 ** 

0.020 

Moderate-risk students 

5.7 

4.6 

1.1 ** 

0.038 

Group meeting for support during a life -changing event 




High-risk students 

3.8 

2.8 

0.9 * 

0.098 

Moderate-risk students 

2.2 

2.2 

0.1 

0.883 

Group meeting about personal goals and behavior 





High-risk students 

5.4 

4.1 

1.4 ** 

0.035 

Moderate-risk students 
Group meeting for social activities 

4.5 

3.4 

1.1 ** 

0.032 

High-risk students 

6.4 

3.2 

3.2 *** 

0.000 

Moderate-risk students 

6.1 

3.1 

^ Q *** 

0.000 

Met with an adult at school to set 





specific goals for the year (%) 
High-risk students 

69.5 

58.1 

114 *** 

0.001 

Moderate-risk students 
Earned rewards for improving grades, 

61.4 

48.4 

13.0 *** 

0.000 

attendance, or behavior, or for reaching 
or making progress toward goals (%) 





High-risk students 

59.2 

56.8 

2.4 

0.503 

Moderate-risk students 

An adult in school connected student to support 

63.1 

57.5 

5.7 ** 

0.047 

programs or help outside of school (%) 
High-risk students 

54.2 

47.3 

6.9 * 

0.058 

Moderate-risk students 

50.4 

38.9 

114 *** 

0.000 

Number of students 1- ’ (N= 1,900) 

947 

953 



High-risk students 0 (N=767) 

406 

361 



Moderate-risk students 0 (N=l,133) 

541 

592 




(continued) 
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Table 4.6 (continued) 


SOURCE: MDRC calculations based on the spring 2013 follow-up student survey. 

NOTES: The analyses reported in this table are based on the student survey analysis sample, which 
includes all students who responded to the spring 2013 follow-up student survey. 

The estimated differences between the case-managed group and the non-case-managed group are 
regression adjusted, controlling for random assignment blocks by school as well as the following 
baseline characteristics: race, gender, free or reduced-price lunch status, English as a second language, 
and whether qualified for a gifted program. 

The values in the column labeled “Case-Managed Group” are the observed means for students 
randomly assigned to the case-managed group. The “Non-Case-Managed Group” values are the 
regression-adjusted means for students randomly assigned to the non-case-managed group, using the 
observed mean covariate values for the case-managed group as the basis for the adjustment. 

High-risk students include those students who were chronically absent, who failed at least one core 
course, or who were ever suspended in the baseline (2011-2012) school year. Moderate-risk students 
include those who were never chronically absent, never failed a core course, and were never suspended 
in the baseline year. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between the case-managed and non-case-managed 
groups. Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

a The times per year measure was created by combining a measure of duration (“For how long did you 
do this activity?”) and a measure of frequency (“How often did you do this activity?”) for each of the 
above support activities in the spring 2013 follow-up student survey. The duration measure was 
converted from a four-point scale to an estimate of the number of weeks per school year a student 
participated in the activity: “1 never did this activity” = 0 weeks, “less than half of the school year” = 9 
weeks, “about half of the school year” =18 weeks, and "most or all of the school year” = 27 weeks. The 
frequency measure was converted from a four-point scale to an estimate of the number of times per 
week a student participated in the given activity: “I never did this activity” = 0, “less than once a month” 
= 0.125, “1-2 times a month” = 0.375, and “one or more times a week” = 1. 

b There is a statistically significant difference (at the 1 0 percent level) between the service contrast 
(“estimated difference”) for the high-risk subgroup and for the moderate -risk subgroup. 

c The sample size reported in the table is for the full student survey analysis sample, minus two school 
districts (193 students) that are excluded because school records data are not available at baseline to 
define the high- and moderate -risk subgroups. The student sample size varies across outcomes due to 
missing data. The percentage of missing data on any given outcome does not exceed 5 percent. 


4.4. Table 4.6 also indicates that there were some activities for which there were no significant 
differences between case-managed and non-case-managed students at either risk level, even 
though Table 4.4 shows an overall difference. These activities include positive behavior 
programs and individual meetings between students and adults related to academic issues. 

Case management appears to be making a more notable difference for students in the 
first category of activities, where the differences are significant even at the smaller sample size 
of each risk group. Although case management is making a difference in the second category of 
activities, that difference is not robust enough to be significant for either risk group. 
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In the third category of activities, there was an overall difference between case- 
managed and non-case-managed students (Table 4.4), but at the risk-group level, there was a 
significant difference for only one risk group (Table 4.6). These are areas where the overall 
difference in supports received seems to be driven more by differences between case-managed 
and non-case-managed students at one of the two risk levels. Moderate-risk case-managed 
students report greater participation than non-case-managed students in individual meetings 
with adults about personal goals and behavior. High-risk case-managed students report greater 
participation than non-case-managed students in community service and volunteering and less 
participation in after-school programs. 

Finally, there are particular support activities for which case management is changing 
students’ engagement at only one of the two risk levels (that is, where Table 4.4 does not show 
an overall difference between case-managed and non-case-managed students, but Table 4.6 
shows a difference at one risk level). Table 4.6 shows that moderate-risk case-managed students 
report earning rewards for accomplishments and high-risk case-managed students report 
participating in college planning activities more than their respective groups of non-case- 
managed peers. Of particular note is that high-risk students overall — case-managed and non- 
case-managed students — report participating in individual meetings with adults related to life- 
changing events more than moderate-risk students; and among the high-risk students, case- 
managed students report participating in these meetings more than the non-case-managed 
students. 

The infonnation presented in Table 4.6 indicates that there was some variation in ser- 
vice receipt for high-risk and moderate-risk students in both the case-managed and non-case- 
managed groups. Taken with Table 4.4, these findings suggest that further investigation may be 
warranted to better understand how students in the high- and moderate-risk groups may be 
differentially targeted for support services. 


Summary and Discussion 

This chapter included infonnation about the students participating in this study and described 
the services received by students assigned to both the case-managed and non-case-managed 
groups. Additionally, it highlighted the key areas of difference between the two groups, ex- 
plored whether the services received varied for high- and moderate-risk students, and raised the 
question of whether the differences in the support services received by case-managed and non- 
case-managed students were large enough to cause differences in their outcomes. 

Baseline characteristics for the study students suggest that those assigned to the case- 
managed and non-case-managed groups were comparable at the time of random assigmnent, 
which supports the notion that random assigmnent created equivalent groups and therefore 


71 



differences between the two groups at the end of the school year are attributable to students’ 
participation in Communities In Schools case management. Overall, the information available 
suggests that there were differences in the support services received by case-managed and non- 
case-managed students — that Communities In Schools case management is providing some- 
thing above and beyond what students otherwise receive. In particular, case-managed students 
reported having more meetings led by adults to discuss academics and behavior and a greater 
proportion of case-managed students reported that an adult in school connected them with out- 
of-school support. This additional attention aligns with the Level 2 service goal of ensuring that 
case-managed students have an adult, or multiple adults, in the school specifically tasked with 
providing support and checking on them throughout the school year. As noted in Chapter 2, 
however, there appear to be many services broadly available to students in these schools, and 
the student surveys confirm that the non-case-managed students participate in a wide variety of 
activities. Even those students not receiving Communities In Schools Level 2 case management 
are getting some support services: They are by no means a “no service” comparison group. 

The findings presented in this chapter also raise the possibility that the level and target- 
ing of services to case-managed students may not be sufficient to improve students’ outcomes 
in this first year of the study. First, the level of services received by case-managed students 
documented in the Communities In Schools MIS data seems lower than might be expected. 
Given that weekly groups and academic support services figured prominently in site coordina- 
tors’ descriptions of the services students receive, one might expect to see a greater number of 
hours and service contacts for case-managed students. On the other hand, given that site 
coordinators have an average of 84 students on their caseloads, 16 hours and 19 contacts may be 
reasonable averages, as this translates to students seeing the site coordinator for approximately 
one school period twice each month they were enrolled. This amount may be sufficient for 
some students, but it is likely that many students are in need of more ongoing and substantial 
support in order to help them stay on track to progress through school. 

Although the MIS data indicate that there was variation in service receipt among case- 
managed students — some students received large amounts of service and others received very 
little — one important question that remains is what drives that variation and whether those 
students who are in need of the most intensive levels of service are receiving the support they 
need. To begin to explore that question, the research team examined differences in levels of 
service receipt for high- and moderate-risk students, as defined using school records from the 
previous school year. While there appears to be a small amount of differentiation in the services 
provided to students in these groups, high-risk case-managed students did not receive a greater 
number of overall hours or service contacts than moderate-risk students — which is especially 
notable given that Communities In Schools sets out to serve the highest-risk students. It may be 
worth further examination into why the students who may be the most likely to drop out of 
school do not seem to receive more intensive levels of case management. Given that high-risk 
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students did not receive a greater number of service contacts or hours of service and that site 
coordinators stated during interviews that some students receive more services because they 
more actively seek out support and interactions with site coordinators, Communities In Schools 
staff members may want to carefully consider whether the students with the greatest needs are 
receiving the attention they need to be successful in school. Alternatively, they may consider 
whether, in some cases, more outgoing students who might otherwise seek different supports 
are receiving disproportionate amounts of site coordinators’ limited time. 

Overall, the critical outstanding question is whether the level and intensity of services 
received by case-managed study students — and the magnitude of the differences that emerged 
between case-managed and non-case-managed students — was enough to shift their academic, 
behavioral, and attendance outcomes in the first year of this study. This question is examined in 
Chapter 5 . 
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Chapter 5 

The Impact of Case Management 


The focus of this chapter is on whether Communities In Schools Level 2 case management had 
an impact on students’ academic and behavioral outcomes. As mentioned in Chapter 1, these 
impact findings are interim, and reflect only one year of students receiving case-managed 
services. A subsequent and final report will present findings on the two-year impact of receiving 
case management. 

As described in Chapter 1, the Communities In Schools experimental study of case 
management investigates impacts on three kinds of student outcomes: primary, secondary, 
and mediating. The two primary outcomes for the evaluation are chronic absenteeism and 
course failure. These two outcomes are primary because they are predictive of students 
staying in school until graduation, which is the ultimate goal of Communities In Schools. 1 
The secondary outcomes for the evaluation are measures of school progress, student behav- 
iors and attitudes, and academic achievement. Impacts on these secondary outcomes are 
examined to aid in the interpretation of the program’s effect on the two primary outcomes 
(course failure and chronic absenteeism). Finally, the mediating outcomes for the evaluation 
are behaviors and attitudes that are precursors to students’ improved success in school: their 
connections to adults, school engagement, attitudes about education, and educational goals 
and expectations (shown in Figure 1.1 as “Mediators”). Examining effects on these mediating 
outcomes can help track the pathway along which Communities In Schools produces impacts 
on the primary and secondary outcomes. 

Random assignment was used to determine which students would participate in Com- 
munities In Schools Level 2 case management, as explained in Chapter 4; therefore, the impact 
of the program can be estimated by comparing the outcomes of students who were invited to 
participate in case management (the case-managed group) with the outcomes of students who 
were assigned not to receive case management, but who retained access to whatever other types 
of services were available in their schools (the non-case-managed group). The tables in this 
chapter show impacts on each outcome and also present the impact estimates as an effect size, a 
common metric that makes it possible to compare the relative size of impacts measured on 
different metrics. (See Box 4.1 for an explanation of how to interpret the findings in the report 
tables and effect sizes.) 


'Given the time frame of this evaluation, the study team cannot track the full sample of middle school and 
high school students through high school graduation. For this reason, the primary outcomes for the evaluation 
are indicators of future school dropout. 
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This chapter starts with a discussion of impacts on mediating outcomes, followed by 
impacts on primary and secondary outcomes. 2 The chapter concludes with a summary section 
that discusses the interim results and explains what to expect in the second-year report. 

Highlights from this chapter include the following: 

• Mediating outcomes. For most of the mediating outcomes — relationships 
with caring, supportive adults at home or school, educational attitudes, 
school engagement, and educational goals and expectations — there were no 
notable differences between students in the case-managed and non-case- 
managed groups. However, based on students’ reports, Communities In 
Schools case management had a positive and statistically significant impact 
on students’ likelihood of having caring, supportive relationships with adults 
outside of home and school, on the quality of their friends and peers, and on 
their belief that education has positive value for their lives. 

• Primary and secondary outcomes. Compared with non-case-managed stu- 
dents, case-managed students had a slightly higher rate of chronic absentee- 
ism and a similar rate of core course failure. There were no significant differ- 
ences between the groups on other measures of school progress, behavior, 
and academic achievement. Thus, after one year, Communities In Schools 
case management has not yet been shown to improve outcomes for students 
related to attendance, course perfomiance, or school discipline. 

Impact on Mediating Outcomes 

As previously shown in the case management logic model (see Figure 1.1 in Chapter 1), the 
mediating outcomes are considered predictors of future outcomes. The expectation is that if 
students experience positive impacts on these outcomes, they will be more likely to experience 
impacts on the subsequent school progress outcomes that relate directly to Communities In 
Schools’ mission to reduce student dropouts. As discussed in Chapter 4, there are eight types of 
direct services case-managed students could receive, with academic, social or life skills, basic 
needs and resources, and enrichment or motivation services being the most frequently adminis- 
tered. According to the Communities In Schools model, being provided with these services 

2 Because the results of the impact analyses given in this report are viewed as interim findings, and not 
confirmatory analyses on the impact of Communities In Schools case management on the primary outcomes of 
chronic absenteeism and course failure after two years, the impact estimates have not been adjusted for 
multiple hypothesis testing. That is, the findings presented here have not been adjusted to account for the fact 
that the greater the number of analyses conducted, the greater the possibility that an individual statistically 
significant finding occurred by chance. 
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should increase the level of attention a student receives, either from a Communities In Schools 
site coordinator or through establishing connections with other adults within and outside the 
school. These additional avenues of support should in turn help students feel more engaged, 
motivated, and connected in school. 

Table 5.1 presents the estimated impact of Communities In Schools Level 2 case man- 
agement on the study’s mediating outcomes, drawn from the survey administered to students in 
the spring of 2013. Overall, there is not a notable pattern of impacts across these measures. The 
first panel of the table relates to students’ reporting that they have relationships with caring 
adults either within the school, at home, or outside of school and home. 3 These three outcomes 
are based on scales rating whether students feel that the adults care, listen, believe in them, and 
encourage them. Both groups of students are similar in terms of having a supportive, caring 
relationship with an adult at home and having a supportive, caring relationship with an adult at 
school, and neither impact estimate is statistically significant. However, there is a statistically 
significant and positive difference between the case-managed and non-case-managed groups in 
whether they have a supportive relationship with an adult outside of home or school (effect size 
= 0.10; p-value = 0.008). 4 This difference suggests that Communities In Schools Level 2 case- 
managed services, which involve identifying community-based supports for students, may be 
connecting the case-managed group with adults outside of the school whom the students are 
able to trust and with whom they form connections. This finding is consistent with the statisti- 
cally significant impact shown in Table 4.4: Case-managed students reported more often than 
non-case-managed students that an adult in school connects students to support or help outside 
of school. 

The next scale reported in Table 5.1 is based on student reports about their friends and 
peers. Items in this scale relate to having caring friends, friends who help out during hard times, 
friends who talk about problems, and friends who do what is right. For this construct, the case- 
managed group reported a slightly higher quality of friends than the non-case-managed group, 
and this estimate is statistically significant (effect size = 0.08; p-value = 0.030). Similar to the 
result about supportive, caring relationships with adults outside of home and school, this result 
suggests that Level 2 case management may help the case-managed students form more trusting 
and supportive relationships with their peers and friends. This impact is consistent with the 


Examples of such adults outside of school and home could include a counselor at a family counseling 
center, a coach for a sports team not affiliated with the school, a staff member from a community organization 
such as a YMCA, or a youth minister at a local church’s after-school program. 

4 Out of 17 tests of difference on mediating outcomes, 3 yielded statistically significant results. Given the 
large number of tests conducted, individual statistically significant findings should be interpreted with caution. 
Where relevant, the authors have noted other study findings that help put the significant findings in context. 
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Evaluation of Communities In Schools 


Table 5.1 


Impacts on Student Behavior and Attitudes for the 
Student Survey Analysis Sample 



Case- 

Non-Case- 


P- Value for 


Managed 

Managed Estimated Effect 

Estimated 

Item 

Group 

Group 

Impact Size 

Impact 

Caring adult at home (l-4) a 

3.36 

3.34 

0.03 0.04 

0.327 

Caring adult at school (l-4) b 

3.23 

3.18 

0.04 0.06 

0.110 

Caring adult outside of home or school (l-4) c 

3.46 

3.37 

0.08 0.10 

*** 0.008 

Friend quality (l-4) d 

2.93 

2.88 

0.05 0.08 

** 0.030 

School-sponsored activities 





Students selecting at least 1 activity (%) 

83.2 

82.1 

1.1 0.03 

0.493 

Mean number of activities done sometimes 

2.8 

3.0 

-0.2 -0.06 

0.135 

Mean number of activities done often 

2.4 

2.4 

0.0 0.00 

0.943 

Non-school-sponsored activities 





Students selecting at least 1 activity (%) 

77.3 

78.9 

-1.6 -0.04 

0.373 

Mean number of activities done sometimes 

1.8 

1.8 

0.0 -0.01 

0.759 

Mean number of activities done often 

1.6 

1.6 

0.0 0.01 

0.895 

Student engagement with school (l-4) e 

2.74 

2.71 

0.03 0.03 

0.333 

Educational attitudes (l-4) f 

3.01 

2.99 

0.02 0.04 

0.217 

Positive educational self-perception and effort® 

2.68 

2.69 

-0.01 -0.02 

0.685 

Negative educational self-perception and effort" 

2.10 

2.06 

0.04 0.05 

0.145 

Positive valuation of education 1 

3.31 

3.24 

0.07 0.11 

*** 0.002 

How far would you like to go in school with your education? (%) 


0.126 

Some high school 

0.9 

1.5 

-0.6 -0.05 


Finish high school 

11.3 

9.8 

1.5 0.05 


Some college or trade/technical school 

5.7 

4.5 

1.3 0.06 


Finish college or trade/technical school 

27.6 

32.0 

-4.4 -0.10 


Graduate school after college 

44.8 

41.9 

2.9 0.06 


Don't know 

9.8 

10.4 

-0.5 -0.02 


How far do you think you will actually go in school or with 




your education? (%) 




0.902 

Some high school 

2.7 

2.5 

0.2 0.01 


Finish high school 

13.4 

13.8 

-0.4 -0.01 


Some college or trade/technical school 

9.3 

9.9 

-0.5 -0.02 


Finish college or trade/technical school 

29.6 

31.0 

-1.5 -0.03 


Graduate school after college 

30.7 

29.4 

1.3 0.03 


Don't know 

14.3 

13.6 

0.7 0.02 


Number of students ' (N=2,093) 

1,058 

1,035 




(continued) 
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Table 5.1 (continued) 


SOURCE: MDRC calculations based on the spring 2013 follow-up student survey. 

NOTES: The analyses reported in this table are based on the student survey analysis sample, which 
includes all students who responded to the spring 2013 follow-up student survey. 

Estimated impacts are regression adjusted using ordinary least squares, controlling for random 
assignment blocks by school, as well as the following baseline characteristics: race, gender, free or 
reduced-price lunch status, English as a second language, whether qualified for a gifted program, and a 
baseline measure of the outcome variable. 

The values in the column labeled “Case-Managed Group” are the observed means for students 
randomly assigned to the case-managed group. The “Non-Case-Managed Group” values are the 
regression-adjusted means for students randomly assigned to the non-case-managed group, using the 
observed mean co variate values for the case-managed group as the basis for the adjustment. 

Effect sizes are calculated by dividing the impact estimate by the standard deviation of the outcome 
measure for students in the student survey analysis sample who are in the non-case -managed group. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between case-managed and non-case-managed groups. 
Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

a Scale based on responses to survey questions 17a-17g, ranging from 1 = “not at all true” to 4 = 
“very much true.” Cronbach's alpha = 0.89. 

b Scale based on responses to survey questions 8a-8f, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach's alpha = 0.87. 

c Scale based on responses to survey questions 15a-15f, ranging from 1 = “not at all true” to 4 = 
“very much true.” Cronbach's alpha = 0.93. 

d Subscale based on responses to survey questions lOa-lOf, ranging from 1 = “not at all true” to 4 = 
“very much true.” Cronbach's alpha = 0.73. 

e Scale based on responses to survey questions 9a-9e, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach's alpha = 0.84. 

f Scale based on responses to survey questions 1 la- 1 In, ranging from 1 = “not at all true” to 4 = 
“very much true.” Cronbach's alpha = 0.84. 

g Subscale based on responses to survey questions 1 lb, lie, Ilf, 1 lg, ranging from 1 = “not at all 
true” to 4 = “very much true.” Cronbach's alpha = 0.73. 

h Subscale based on responses to survey questions 11a, lie, lid, 1 lh, ranging from 1 = “not at all 
true” to 4 = “very much true.” Cronbach's alpha = 0.75. 

‘Subscale based on responses to survey questions 1 lj-1 In, ranging from 1 = “not at all true” to 4 = 
“very much true.” Cronbach's alpha = 0.79. 

'The sample size reported in the table is for the full student survey analysis sample. However, the 
sample size varies across outcomes due to missing data. The percentage of missing data on any given 
outcome does not exceed 1 percent. 


finding presented in Chapter 4 that case-managed students reported being more involved in peer 
group meetings led by adults, a setting in which supportive peer relationships could have been 
cultivated. 

Students also responded to survey questions about their frequency of participation in 
school-sponsored and non-school-sponsored activities, such as school sports teams, art or music 
groups, academic clubs, Junior ROTC, or cultural groups. The differences in the percentages of 
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case-managed and non-case-managed students who participated in at least one school- 
sponsored activity or in at least one non-school-sponsored activity were less than 2 percentage 
points in each case and not statistically significant. In addition, case-managed and non-case- 
managed students participated in activities with similar frequency. 

The items within the school engagement scale shown in Table 5.1 generally asked 
whether students feel happy, feel safe, feel that they’re treated fairly, and feel like a part of their 
school. The items within the educational attitudes scale asked students about their own percep- 
tions of whether they do well at school, plan their work, persist with homework and school- 
work, give up easily, or have trouble figuring out answers in school. There are no statistically 
significant differences between case-managed students’ and non-case-managed students’ 
reports about their engagement with school and overall educational attitudes. 

Table 5.1 shows three subscales related to educational attitudes. The first subscale fo- 
cused on students’ positive educational self-perception and effort. The items in this scale asked 
students about whether they believe they do well at school, feel that they are as smart as other 
students, and are persistent with homework and study plans. The second subscale describes a 
similar construct, but relates to negative educational self-perceptions: for example, whether 
students think they are slow in finishing their homework, have trouble figuring out answers, or 
easily give up if a task is hard. There is no notable or statistically significant difference between 
case-managed and non-case-managed groups in their positive and negative educational self- 
perceptions. The third educational attitude subscale is about students’ positive valuation of 
education, including items about whether students believe that education will be valuable to get 
a job, whether it is important to get good grades, and whether school is useful in making good 
decisions in life. For this subscale there is a statistically significant difference between the case- 
managed and non-case-managed groups (effect size = 0.11; p-value = 0.002), which suggests 
that the supports students receive through Communities In Schools case management may be 
helping them value education more and understand how it can influence their future path. 

Table 5.1 also shows the impact of Communities In Schools Level 2 case-managed ser- 
vices on students’ self-reported academic goals and expectations. An omnibus test of statistical 
significance across the goal and expectation categories shows that there is no difference overall 
between the case-managed and non-case-managed group with regard to their reported educa- 
tional goals — how far they would like to go in their education — and educational expectations 
— how far they think they actually will go with their education. While case-managed students 
see greater value in their education than their non-case-managed peers, this did not carry over to 
their self-reported educational goals and expectations. 

As explained above, the mediating outcomes are theorized to be a bridge between stu- 
dents receiving services — and consequently experiencing changes in their connections, 
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supports, and attitudes — and impacts on the study’s main outcomes that relate to Communities 
In Schools’ driving mission to keep students on track to graduation. That is, these attitudinal and 
engagement-oriented outcomes are seen as predictive of later school-related outcomes. The next 
section of this chapter discusses impact results on such school outcomes. 


Impact on Primary and Secondary Outcomes 

Table 5.2 presents the estimated impact of Communities In Schools Level 2 case-managed 
services on the primary and secondary student outcomes after one year of the study. The 
primary outcomes relate to factors that impede a student’s progress toward graduation: course 
failure and chronic absenteeism. The secondary outcomes relate to additional aspects of course 
performance and attendance, as well as student behavior, that are predictive of students making 
it through the K-12 education system to graduation. (Please note that in Table 5.2 the desired 
effect may be either positive or negative. For chronic absenteeism, course failure, and average 
number of suspensions, a negative impact estimate value would indicate fewer of these prob- 
lems for case-managed students than for non-case-managed students — better outcomes for the 
case-managed group.) The results in Table 5.2 for the two primary outcomes show that a 
marginally higher percentage of case-managed students — 3 percentage points more — were 
chronically absent compared with non-case-managed students (effect size = 0.09; p-value = 
0.056). About 2 percentage points more case-managed students than non-case-managed 
students failed a core course. The magnitude of the effect is very small and not statistically 
significant. In terms of secondary outcomes, the one school progress outcome, core credit 
accumulation toward graduation, reflects the percentage of core credits accumulated to meet the 
state graduation requirement for core classes. To graduate from high school in four years, a 
student would need to earn an average of 25 percent of required core credits annually. The 
results show that case-managed and non-case-managed groups in high school accumulated 
almost the same percentage of the core course credits required for graduation (about 20 per- 
cent). The impacts of Communities In Schools case management on the outcomes in the student 
behavior domain, attendance and number of suspensions, were small and not statistically 
significant. Finally, the results are similar for both groups of students regarding academic 
achievement as measured by average course marks — 79 out of 100 points, or about a B-/C+ 
average mark. 5 


5 The study team also analyzed the impacts on these primary and secondary outcomes for the high- and 
moderate-risk subgroups that were included in analyses reported in Chapter 4. The findings were similar to the 
overall findings. Among the moderate-risk student group there were no statistically significant differences on 
any of the outcomes listed in Table 5.2. Among the high-risk student group, there was one statistically 
significant difference similar to the full student sample: A higher percentage of case-managed students were 
chronically absent compared with non-case-managed students. 
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Evaluation of Communities In Schools 
Table 5.2 


Impacts on Academic and Behavioral Outcomes for the 
School Records Analysis Sample 


Outcome 

Case- 

Managed 

Group 

Non-Case- 
Managed Estimated 
Group Impact 

Effect 

Size 

P- Value for 
Estimated 
Impact 

Primary outcomes 

Chronic absenteeism 11 (%) 

17.92 

14.83 

3.09 

0.09 * 

0.056 

Failed at least 1 core course (%) 

32.26 

30.24 

2.03 

0.04 

0.270 

School progress’ 1 

Core credit accumulation for graduation (%) 

20.28 

20.42 

-0.15 

-0.02 

0.768 

Student behavior 

Average attendance rate (%) 

93.91 

94.27 

-0.36 

-0.05 

0.220 

Number of suspensions 

1.46 

1.29 

0.16 

0.06 

0.164 

Academic achievement 

Average core course marks (%) 

79.33 

79.75 

-0.42 

-0.05 

0.118 

Sample size 0 (N=2,048) 

1,029 

1,019 





SOURCE: MDRC calculations based on student records obtained from school districts. 

NOTES: The analyses reported in this table are based on the school records analysis sample, which 
includes all students with course failure data for the 2012-2013 school year (one study district is 
excluded from this sample because student records data for this school year are not available). 

Estimated impacts are regression-adjusted using ordinary least squares, controlling for random 
assignment blocks by school, as well as the following baseline characteristics: race, gender, free or 
reduced-price lunch status, English as a second language, whether qualified for a gifted program, and a 
baseline measure of the outcome variable. 

The values in the column labeled “Case-Managed Group” are the observed means for students 
randomly assigned to the case-managed group. The “Non-Case-Managed Group” values are the 
regression-adjusted means for students randomly assigned to the non-case-managed group, using the 
observed mean co variate values for the case-managed group as the basis for the adjustment. 

Effect sizes are calculated by dividing the impact estimate by the standard deviation of the outcome 
measure for students in the school records analysis sample who are in the non-case-managed group. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between case-managed and non-case-managed groups. 
Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

a A student is considered chronically absent if he or she has an attendance rate below 90 percent. 

b This outcome is for high school students only; the state-level graduation requirements for the four 
core subjects are used in this calculation. A student can be expected to accumulate about 25 percent of 
core credits in each year of high school. 

c The sample size reported in the table is for the full school records analysis sample. However, the 
sample size varies across outcomes due to missing data. The percentage of missing data is 19 percent for 
attendance rates and absenteeism, 6 percent for suspensions, 1 percent for credit accumulation, and 0 
percent for other outcomes. 
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Summary and Discussion 

At this interim stage, Communities In Schools Level 2 case-managed services in the first year of 
the study — the 2012-2013 school year — had a small impact on chronic absenteeism: a higher 
percentage of case-managed students than non-case-managed students were chronically absent 
from school (that is, they had attendance rates of 90 percent or lower). Case management did 
not have a statistically significant impact on the percentage of students failing at least one core 
course. Case management did not lead to notable differences in average attendance rate, credit 
earning, course performance, or suspensions between the case-managed students and their non- 
case-managed counterparts. In general, the mediating outcomes of case-managed and non-case- 
managed students were also similar. However, the positive and statistically significant impacts 
on friend quality and students’ relationships with caring adults outside of home or school 
suggest that Communities In Schools Level 2 case-managed services may be starting to help 
students form more trusting connections and networks of support with peers and adults. The 
impacts on these outcomes are consistent with implementation findings reported in Chapter 4 
that case-managed students were more involved in several types of organized support run by 
adults, particularly for groups of students, compared with their non-case-managed peers. The 
implementation research also found that more case-managed students reported being comiected 
to programs outside of school for support. 

As explained in the first chapter, this is an interim report, the first of two reports inves- 
tigating the implementation and impact of Communities In Schools Level 2 case-managed 
services. The next report will present the impact of case management on student outcomes after 
two years — the 2012-2013 and 2013-2014 school years. Some of the findings in this report 
suggest theories for why case management might or might not improve the outcomes of 
students after two years. The participating study schools tend to have many and various services 
available to students, and non-case-managed students have access to many of the same kinds of 
services to which case-managed students do. In some cases, those services may even be Level 1 
services provided or coordinated by Communities In Schools. In these school environments, the 
case management provided by site coordinators may not be able to create supportive experienc- 
es that are different enough from the widely available services to result in improved outcomes 
for students on their caseloads. 

Additionally, there does not appear to be much service differentiation between high-risk 
and moderate-risk case-managed students. Given that the high-risk students are more likely to 
have ongoing struggles with absenteeism, behavior, and school perfonnance, and that they do 
not seem to consistently receive more intensive services, it may prove difficult to shift these 
students onto a more positive educational trajectory. Finally, although the differences between 
the case-managed and non-case-managed students on the primary and secondary outcomes are 
not statistically significant (with one exception), they all suggest slightly better outcomes for the 
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non-case-managed group on average. This consistency favoring the non-case-managed group 
suggests that another year of case management may not be enough to have a positive effect on 
student outcomes. 

However, it could be that a second year of case management will yield positive impacts 
for students. Students and site coordinators may need time working together for student behav- 
iors and attitudes to shift, and, ultimately, for student outcomes to improve. Site coordinators 
may also need more time to remove nonacademic or structural barriers to students’ success (for 
example, helping resolve a complicated home situation). More time may allow site coordinators 
to strengthen their relationships with the students on their caseloads and learn more about the 
extent and nature of their needs, allowing them to better target services and calibrate their 
intensity. Furthermore, as discussed earlier in this section, there is evidence that case-managed 
students, in comparison with non-case-managed students, are forming more supportive connec- 
tions with both adults and peers. If these relationships are sustained for more than one school 
year, they could result in improved outcomes for these students. It may also be that students 
need to reach and pass service receipt thresholds for the benefits of those services to emerge, 
which may take more time; one year of support may simply not be enough. Some research on 
other student support programs that found no impacts or negative impacts on student outcomes 
like attendance and academic performance after one year did find positive impacts after two or 
more years. Among the hypotheses for why these impacts emerged later was that students 
needed to adjust to the higher expectations being placed on them. 6 


Suggestions for Continuous Improvement 

Although the evaluation study of Communities In Schools Level 2 case management is ongo- 
ing, the research about one year of case management services suggests some areas where 
Communities In Schools may want to consider change and to which similar service providers 
should be attentive in planning their delivery of services. 

• Overall, the implementation research suggests that the participating schools 
and affiliates follow the same steps in the case management process — a 
noteworthy finding given Communities In Schools’ expansive national net- 
work. However, these schools and affiliates varied in how they implemented 
each step. In particular, site coordinators’ ongoing assessment of students’ 
needs and their use of data collected as part of the monitoring process may 
benefit from greater consistency across schools. Communities In Schools’ 

6 For example, see the evaluations of the Higher Achievement Program (Herrera, Grossman, and Linden 
2013) and AVID (Dunn et al. 2010), as well as the summary report about integrated student services from 
Moore et al. (2014), which suggests that the impact of such services may need time to emerge. 
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national office could consider suggesting best practices for the network re- 
garding how and with what kind of data site coordinators assess students’ 
ongoing needs throughout the year. In addition, site coordinators may benefit 
from additional support regarding how to use these ongoing assessment data 
to ensure that services are continually adjusted to address students’ needs and 
increase the likelihood of improvements in student outcomes. 

• Recognizing that some students have more intensive needs than others, 
Communities In Schools may want to develop additional guidelines regard- 
ing the relationship between levels of service and student needs. Even though 
site coordinators indicated in interviews that they varied the level of attention 
paid to different students based on their perceived needs, the analyses of ser- 
vice receipt in this study showed considerable similarity in the frequency and 
kinds of services received by high-risk and moderate-risk students. Site coor- 
dinators might benefit from more guidance on assessing levels of student risk 
or need and identifying appropriate levels of service in response. The impli- 
cation is that site coordinators would focus more time and energy on the most 
struggling students, making sure that they get adequate attention, and spend 
less time with the moderate-risk students. Such service differentiation, if 
standardized within the Communities In Schools model, would in effect 
result in a three-level service model. 

• The schools in this study have a range of services in place to help students be 
more successful, including broadly available Level 1 service coordination by 
Communities In Schools site coordinators. In such schools, it may be more 
challenging for Level 2 case management to make a difference above and 
beyond the services (Level 1 or otherwise) that already exist. Therefore, it 
may be beneficial for Communities In Schools to consider where it can add 
the greatest value in each school and how that may change over time. Per- 
haps in schools with many services already available to students, Communi- 
ties In Schools should focus their efforts on providing Level 2 case-managed 
services only to the students most in need and focus much less, if at all, on 
Level 1 services. In schools with relatively few school-wide supports, Com- 
munities In Schools may be able to add substantial value by having site coor- 
dinators spend more time on Level 1 services. The Communities In Schools 
national office may be in a position to provide guidance to affiliates on find- 
ing an appropriate balance between Level 1 and Level 2 services so that the 
program can maximize its value in each school. 
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Next Steps for the Study of Case Management 

The research activities that are part of this study have continued, and the results from ongoing 
analyses will be shared in a second report. The next report will build on and complement this 
report in three ways: 

• Two-year impacts. Similar student data on primary, secondary, and mediat- 
ing outcomes are being collected for the 2013-2014 school year. The analysis 
of these data will result in two-year impact findings that better assess the 
effectiveness of case management, given that most students on a site coordi- 
nator’s caseload receive case-managed services for more than one year. Pre- 
liminary data suggest that about two-thirds of the 2012-2013 case-managed 
students in our sample continued to receive case management in 2013-2014. 

• Additional implementation findings. The second report will include more 
implementation infonnation, creating an opportunity to see whether the case 
management experience for students changed over the course of two years. It 
will again include information about service provision and receipt, as well as 
the contrast between the services accessed by case-managed and by non- 
case-managed students. New implementation data collected during the 2013- 
2014 school year will make it possible for the next report to discuss the 
alignment of the types of services provided to students with the specific 
needs students have. It will discuss the roles of Communities In Schools’ 
community partners and the nature of their partnerships with the local affili- 
ates and school sites. To further understand the contrast that site coordinator 
case management might make in student service provision, the report will 
present more infonnation about the work of guidance counselors and social 
workers, the school staff members most comparable to Communities In 
Schools site coordinators in the work that they do. 

• Variation. Furthennore, the next report will investigate variation across 
school sites in tenns of both implementation and impacts and the associations 
between the two. This analysis may generate lessons about what kinds of 
contexts and implementation characteristics are associated with positive im- 
pacts on student outcomes. 

The additional findings in the next report may also lead to further programmatic con- 
siderations for Communities In Schools and other integrated student service providers. 
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Appendix A 

Statistical Model and Statistical Power 




This appendix discusses various technical issues related to the estimation of program impacts. 
The first section provides the statistical model used to estimate the impact of Communities In 
Schools case management on student outcomes. The second section discusses the minimum 
detectable effect size for the main impact findings in the study. The final section presents impact 
estimates that are not adjusted for students’ baseline characteristics. 


Statistical Model for Estimating Impacts 

The impact of Communities In Schools case management on student outcomes is estimated by 
fitting the following regression model to the relevant analysis sample (the school records 
analysis sample or the student survey analysis sample): 

Yi = m + Y. K X k B ki + lsS s X si +Y s oj s M si + a h ( 1 ) 

where: 

Yj = the outcome of interest for student i 

Tj = one if student i was assigned to the case-managed group and zero otherwise 

B kl = a set of K random assigmnent block indicators, equal to one if student i is in ran- 
dom assignment block k and zero otherwise 

X si = a set of S baseline characteristics for student i 

Mi = a set of S missing indicators for each of the student characteristics, coded one if 
missing and zero otherwise 

e i = a within-student error term. 

Therefore: 

P = the estimated impact of case management on outcome Y. 

The block indicators are included in the model to capture a central feature of the re- 
search design in which random assignment was conducted separately for each school . 1 
Controlling for random assigmnent blocks in the model also accounts for the clustering of 


’in one site, random assignment was also conducted by grade level and gender because a specific number 
of boys and girls in each grade had to be served. In total, there are 34 random assignment blocks in the full 
study sample and the student survey analysis sample, and 31 blocks in the school records analysis sample. 
There are fewer blocks in the latter sample because school records data are not available for one school district 
(and therefore this district is excluded from the sample). 
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student outcomes by school, because it explains all of the between-school variation in student 
outcomes. 2 

Controlling for students’ baseline characteristics is not necessary for obtaining unbiased 
impact estimates, because random assignment should ensure that the program and control 
groups have similar observed and unobserved characteristics at baseline. 3 However, controlling 
for student characteristics can increase the precision of the impact estimates, because these 
characteristics explain part of the within-block variation in the outcome measure. Controlling 
for student characteristics can also be used as a “safeguard” to ensure that the treatment and 
control group are comparable on all characteristics. 4 (As a point of reference, unadjusted impact 
estimates are presented later in this appendix.) 

The statistical significance of impact estimates (and other estimates) in this report is 
assessed using a two-tailed t-test. Statistical significance is a measure of the degree of certainty 
that one may have that a program’s impact is actually nonzero. If an impact estimate is statisti- 
cally significant, then one may conclude with some confidence that the program really had an 
effect on the outcome being assessed. If an impact estimate is not statistically significant, then 
the nonzero estimate is more likely to be a product of chance. In this report, statistical signifi- 
cance is based on a significance level of 10 percent. 

Finally, it is important to note that the impact estimates presented in this report are “in- 
tent to treat” estimates of the effect of C ommunities In Schools case management. About one in 
ten students assigned to case management did not receive the intended services. Thus, the 
findings in this report represent the estimated impact of offering case management to students 
(“intent to treat”), rather than the impact of case management on students who actually received 
the intended services (“treatment on the treated”). Because students’ participation in educational 


2 The random assignment ratio differs across blocks (minimum = 0.27, maximum = 0.72, median = 0.51, in 
the full study sample). These differences in the random assignment ratio must be accounted for to obtain an 
unbiased estimate of impacts. There are several ways to account for variation in the random assignment ratio. 
The two most common are to (a) “block-mean” center the covariates on the right-hand side of the model or 
(b) include block fixed effects in the model. Raudenbush (2009) shows that these two methods produce the 
same impact estimate. This model is based on the latter approach. 

3 The following covariates are included in the statistical model: whether the student has English as a second 
language (ESL), whether the student is qualified for a gifted program, whether the student is eligible for free or 
reduced-price lunch, the student’s race/ethnicity and gender, and a baseline measure of the outcome variable. 
These covariates were chosen because they are strong predictors of academic achievement; the decision about 
which covariates to include in the model was made before starting the impact analysis. 

4 ln particular, when differences between the treatment and control group are between 0.05 and 0.25 stand- 
ard deviations (as they are in this study; see Appendix B), the What Works Clearinghouse recommends that 
these characteristics be included as covariates in the impact model (What Works Clearinghouse 2014). 
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interventions is typically voluntary, intent-to-treat estimates of the impact of offering a program 
or service are relevant to policy. 5 


Minimum Detectable Effect Sizes 

This section examines how large the impact of Communities In Schools case management 
would have to be for the evaluation to be able to detect it. A common way to convey a study’s 
statistical power is through the minimum detectable effect (MDE) or the minimum detectable 
effect size (MDES). Formally, the MDE is the smallest true program impact that can be detect- 
ed with a reasonable degree of power (in this case, 80 percent) for a given level of statistical 
significance (in this case, 10 percent for a two-tailed test). The MDES is the MDE scaled as an 
effect size — in other words, it is the MDE divided by the standard deviation of the outcome of 
interest. Effect sizes are used widely for measuring the impacts of educational programs and are 
defined in terms of the underlying population’s standard deviation of student achievement. For 
example, an MDES of 0.20 indicates that an impact estimator can reliably detect a program- 
induced increase in student achievement that is equal to or greater than the 0.20 standard 
deviation of the existing student distribution. 

The MDE and MDES for a study are a function of the standard error of the estimated 
program impact: 6 

MDE = M N _ B _ x *s.e.(P) (2a) 

MDES = M N _ B _ X * (2b) 

<j 

where: 

s.e.(P ) = the standard error of the impact estimate 

c r = the standard deviation that is used to calculate effect sizes (for example, in 

this study, it is the standard deviation for the non-case-managed group) 


5 The estimated effect of the “treatment on the treated" in the student survey analysis sample can be ob- 
tained by dividing the "intent to treat" impact estimates presented in this report by 86.8 percent, which is the 
difference (after rounding) in the percentage of students in the case-managed group who actually received 
services (88.9 percent) and the percentage of students in the non-case-managed group who received case 
management services (2 percent). For the student records analysis sample, the divisor is 88.8 percent, calculat- 
ed from the corresponding figures of 90.7 percent and 1.9 percent, respectively. See Bloom (2006) for a 
discussion. 

6 This is because the standard error of the impact estimate is what determines whether the impact estimate 
is statistically significant. 
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N 

B 

X 


= the number of students in the sample 

= the number of random assigmnent blocks in the impact analysis 

= the number of student baseline characteristics and missing-data indicator 
variables included as covariates in the impact model (see previous section) 


M N -B-x = the “degrees of freedom” multiplier, which is calculated to be 2.5 in this 
study, assuming a two-tailed test with a statistical power level 0.80 and a 
statistical significance level of 0.10. 


Appendix Table A.l presents the MDES for the school records outcomes in this report, 
including the two primary outcomes (chronic absenteeism and course failure). As shown in this 
table, the study is able to detect an effect of 4 percentage points on chronic absenteeism (an 
effect size of 0. 1 1) and an effect of 4.6 percentage points on the percentage of students who fail 
at least one course (an effect size of 0.10). 


Unadjusted Impact Estimates 

As explained earlier, the statistical model used to estimate impacts controls for several measures 
of students’ baseline characteristics and prior achievement (see Equation 1). Although it is not 
strictly necessary to control for these baseline characteristics when using a random assigmnent 
design, the main impact analysis does so in order to improve the precision of the impact 
estimates. Controlling for students’ baseline characteristics should not appreciably affect the 
estimated impact — but it should reduce its standard error. 

To confirm that this is true, Appendix Table A.2 compares the estimated impacts on 
school records outcomes from Chapter 5 (which are adjusted for student baseline characteris- 
tics) with impact estimates that are adjusted for blocking only (not adjusted for student charac- 
teristics). The table also shows the standard error of these impact estimates. As expected, 
controlling for student characteristics does not affect the magnitude of the impact estimates, but 
it does decrease their standard error. The extent to which controlling for student characteristics 
reduces the standard error varies across outcomes — from a reduction of 0. 1 percent in the 
standard error for the impact on chronic absenteeism to a reduction of 2.1 percent in the stand- 
ard error average for the impact on the average attendance rate. 
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Evaluation of Communities In Schools 
Appendix Table A.l 

Minimum Detectable Effect and Effect Size 
for Impacts on Academic and Behavioral Outcomes for the 
School Records Analysis Sample 

Number of 

Outcome Students MDE MDES 

Primary outcomes 
Chronic absenteeism 3 (%) 

Failed at least 1 core course (%) 

School progress” 

Core credit accumulation for graduation (%) 865 1.24 0.15 

Student behavior 
Average attendance rate (%) 

Suspensions (#) 

Academic achievement 

Average core course marks (%) 2,048 0.67 0.09 

SOURCES: MDRC calculations based on student records obtained from school districts. 

NOTES: The analyses reported in this table are based on the school records analysis sample, 
which includes all students with course failure data for the 2012-2013 school year (one study 
district is excluded from this sample because student records data for this school year are not 
available). 

The minimum detectable effect (MDE) and minimum detectable effect size (MDES) in this 
table are calculated based on the standard error of the impact estimate (adjusted for random 
assignment blocks and student baseline characteristics) and the number of students in the school 
records analysis sample. A statistical significance level of 10 percent is assumed. The MDES is 
calculated by dividing the MDE by the standard deviation of the outcome measure for students in 
the school records analysis sample who are in the non-case-managed group. 

a A student is considered chronically absent if he or she has an attendance rate below 90 
percent. 

b This outcome is for high school students only; the state-level graduation requirements for the 
four core subjects are used in this calculation. A student can be expected to accumulate around 25 
percent of core credits in each year of high school. 


1,667 0.74 0.10 

1,935 0.29 0.10 


1,667 4.01 0.11 

2,048 4.57 0.10 
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Appendix Table A.2 


Estimated Impacts on Student Academic and Behavorial Outcomes for the School Records 
Analysis Sample, Adjusted and Unadjusted for Student Characteristics 



Adjusted for Blocking and 
Full Set of Student Characteristics 2 

Adjusted for Blocking Only 

Outcome 

Estimated 

Impact 

(S.E.) 

Effect 

Size 

P- Value for 
Estimated 
Impact 

Estimated 

Impact 

(S.E.) 

Effect 

Size 

P-Value for 
Estimated 
Impact 

Primary outcomes 

Chronic absenteeism 15 (%) 

3.09 * 
(1.614) 

0.09 

0.056 

3.03 * 
(1.616) 

0.08 

0.061 

Failed at least 1 core course (%) 

2.03 

(1.837) 

0.04 

0.270 

1.95 

(1.840) 

0.04 

0.289 

School progress' 

Core credit accumulation for graduation (%) 

-0.15 

(0.497) 

-0.02 

0.768 

-0.28 

(0.496) 

-0.03 

0.572 

Student behavior 

Average attendance rate (%) 

-0.36 

(0.296) 

-0.05 

0.220 

-0.29 

(0.302) 

-0.04 

0.330 

Number of suspensions 

0.16 

(0.116) 

0.06 

0.164 

0.15 

(0.118) 

0.05 

0.190 

Academic achievement 

Average core course marks (%) 

-0.42 

(0.270) 

-0.05 

0.118 

-0.43 

(0.273) 

-0.06 

0.114 

Sample size d 

2048 



2048 
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Appendix Table A.2 (continued) 


SOURCE: MDRC calculations based on student records obtained from school districts. 

NOTES: The analyses reported in this table are based on the school records analysis sample, which includes all students 
with course failure data for the 2012-2013 school year (one study district is excluded from this sample because student 
records data for this school year are not available). 

All estimated impacts are regression adjusted using ordinary least squares, controlling for random assignment blocks 
by school and grade level. 

“S.E.” indicates standard error, given in parentheses. 

Effect sizes are calculated by dividing the impact estimate by the standard deviation of the outcome measure for 
students in the school records analysis sample who are in the non-case-managed group. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between case-managed and non-case-managed groups. Statistical 
significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimated impacts are adjusted for blocking and the following variables: race, gender, free or reduced-price lunch 
status, English as a second language, whether qualified for a gifted program, and a baseline measure of the outcome 
variable. 

b A student is considered to be chronically absent if he or she has an attendance rate below 90 percent. 

c This outcome is for high school students only; the state-level graduation requirements for the four core subjects are 
used in this calculation. A student can be expected to accumulate around 25 percent of core credits in each year of high 
school. 

d The sample size reported in the table is for the full school records analysis sample. However, the sample size varies 
across outcomes due to missing data. The percentage of missing data is 19 percent for attendance rates and absenteeism, 6 
percent for suspensions, 1 percent for credit accumulation, and 0 percent for other outcomes. 




Appendix B 

Sample and Response Analysis 




This appendix provides additional information about varying samples of students and response 
rates. The three exhibits for this appendix show how the different student analysis samples were 
formed, provide baseline characteristics for the full study sample, and give response rates for 
both the case-managed and non-case-managed students based on information source. 

Appendix Figure B.l shows how the original pool of students recruited for participa- 
tion in the evaluation was reduced to the samples of students whose data were analyzed in this 
report. The eligibility pool of 2,578 represents all recruited students whose parents consented 
to their participation in Communities In Schools case management. 1 To provide all students 
with the same opportunity to receive Communities In Schools case-managed services, MDRC 
implemented a random assignment process that assigned students to one of three groups: a 
case-managed group to fill open caseload slots in participating schools (1,179 students), a 
non-case-managed control group (1,118 students), and a “wait-list” group (281 students who 
could fill caseload slots that might open up during the course of the school year but who 
would not participate in the research activities and not be included in any analyses). Sixty- 
seven of the students assigned to the first two groups were removed from the study sample 
either because they had consented to participate only in Communities In Schools case man- 
agement and not in the evaluation or because they had exited their participating school after 
having submitted a consent form but before random assignment had been conducted. 2 This 
left a study sample of 1,140 case-managed students and 1,090 non-case-managed students, or 
a total of 2,230 students. 

Of this student sample, many students responded to surveys administered at the end of 
the 2012-2013 school year: the student survey analysis sample. Of the 2,230 students in the 
study sample, only 137 students did not respond to the survey. Also, school records data was 
obtained for many of the students in the study sample: the school records analysis sample. The 
study team was unable to obtain records data for only 1 82 of the study students; most of these 
students were from one school district from which the study team was unable to obtain the 
necessary data for this report. There is a great amount of overlap of these two analysis samples: 
88.7 percent of the study sample is represented in both analysis samples, including 86.8 percent 
of case-managed students and 90.6 percent of non-case-managed students. 

Appendix Table B.l presents the baseline characteristics of the 2,230 students in the full 
study sample. Similar to the information provided in the report about the analysis samples, the 


'Students who were 18 years of age or older — adults — were able to sign consent for themselves and did 
not need parental consent to participate. 

‘Although these students exited schools before random assignment, the research team was not informed 
about their exits until after random assignment occurred. 
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Appendix Figure B.l 
Creation of the Student Analysis Samples 
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Appendix Table B.l 

Baseline Characteristics of Students in the 
Full Study Sample 


Characteristic 

Case- 

Managed 

Group 

Non-Case- 
Managed Estimated 
Group Difference 

Effect 

Size 

P-Value for 
Estimated 
Difference 

Race/ethnicity (%) 





0.269 

Hispanic 

60.8 

60.0 

0.8 

0.02 


Black, non-Hispanic 

34.8 

33.6 

1.2 

0.03 


White, non-Hispanic 

2.1 

3.2 

-1.1 

-0.07 


Asian 

1.3 

1.9 

-0.6 

-0.05 


Other 

0.9 

1.3 

-0.4 

-0.04 


Male (%) 

45.0 

44.4 

0.6 

0.01 

0.759 

Eligible for free/reduced-price lunch (%) 

54.6 

52.2 

2.4 

0.05 * 

0.078 

English as a second language (%) 

11.8 

11.3 

0.5 

0.01 

0.722 

Student qualifies for gifted program (%) 

2.8 

3.4 

-0.6 

-0.03 

0.422 

Chronically absent (%) 

9.9 

9.3 

0.6 

0.02 

0.644 

Average attendance rate (%) 

95.7 

95.7 

0.0 

-0.01 

0.843 

Failed at least 1 core course (%) 

26.3 

26.2 

0.2 

0.00 

0.932 

Average core course marks (%) 

80.0 

80.3 

-0.3 

-0.04 

0.427 

Household makeup 3 (%) 






Lives with 2 or more parents/guardians 

54.4 

51.8 

2.6 

0.05 

0.220 

Lives with 1 parent/guardian 

39.0 

40.7 

-1.7 

-0.03 

0.427 

Lives with 1 or more grandparent(s) 

7.4 

9.1 

-1.7 

-0.06 

0.161 

Lives with his/her own child 

2.6 

2.5 

0.1 

0.01 

0.909 

Language predominantly spoken at home b (%) 






English 

76.8 

76.6 

0.1 

0.00 

0.940 

Not English 

23.2 

23.4 

-0.1 

0.00 

0.940 

Parent educational attainment (%) 






Father 





0.523 

Not a high school graduate 

17.7 

20.7 

-3.1 

-0.08 


High school graduate or GED recipient 

25.9 

22.8 

3.1 

0.07 


College graduate or higher 

12.0 

13.9 

-2.0 

-0.06 


Don't know 

44.5 

42.5 

2.0 

0.04 


Mother 





0.762 

Not a high school graduate 

18.3 

21.0 

-2.8 

-0.07 


High school graduate or GED recipient 

28.3 

28.2 

0.1 

0.00 


College graduate or higher 

23.6 

23.8 

-0.2 

-0.01 


Don't know 

29.9 

27.0 

2.9 

0.06 



(continued) 
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Appendix Table B.l (continued) 



Case- 

Non-Case- 


P-Value for 


Managed 

Managed 

Estimated Effect 

Estimated 

Characteristic 

Group 

Group Difference Size 

Difference 

Did any siblings leave high school before 
graduation? 0 (%) 





None left high school 

63.9 

65.7 

-1.8 -0.04 

0.483 

At least 1 left high school 

36.1 

34.3 

1.8 0.04 

0.483 

Student engagement with school ( 1 -4) d 

2.84 

2.87 

-0.03 -0.04 

0.370 

How far would you like to go in school with 
education? (%) 

your 



0.860 

Some high school 

0.6 

1.2 

-0.5 -0.05 


Finish high school 

12.0 

9.3 

2.7 0.09 


Some college or trade/technical school 

6.4 

5.7 

0.7 0.03 


Finish college or trade/technical school 

44.5 

45.1 

-0.7 -0.01 


Graduate school after college 

29.7 

31.6 

-1.9 -0.04 


Don't know 

6.8 

7.2 

-0.4 -0.01 


How far do you think you will actually go in school 




or with your education? (%) 




0.733 

Some high school 

1.8 

1.0 

0.8 0.08 


Finish high school 

12.8 

12.7 

0.1 0.00 


Some college or trade/technical school 

10.9 

10.7 

0.1 0.00 


Finish college or trade/technical school 

38.1 

40.9 

-2.8 -0.06 


Graduate school after college 

25.0 

25.8 

-0.8 -0.02 


Don't know 

11.5 

8.9 

2.5 0.09 


Caring adult at home ( 1 -4) e 

3.38 

3.40 

-0.03 -0.05 

0.246 

Caring adult at school (1-4) 
Caring adult outside of home 

3.30 

3.28 

0.02 0.03 

0.516 

or school (l-4) g 

3.46 

3.49 

-0.03 -0.05 

0.249 

Joint test of difference between groups 




0.985 

Sample size 11 (N = 2,230) 

1,140 

1,090 




(continued) 


case-managed and non-case-managed students are highly similar across all the measured charac- 
teristics, as indicated by the p-value of 0.985 for the overall test of difference. Also, the case- 
managed and non-case-managed groups’ values on these baseline characteristics are very similar 
to the values presented separately for each analysis sample in Tables 4. 1 and 4.2 of the report. 

Appendix Table B.2 compares the response rates of the case-managed and non-case- 
managed students on the survey and gives the percentage of students in each of those two 
groups for whom school records data were obtained. Overall, the rates of response for these two 
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Appendix Table B.l (continued) 


SOURCE: MDRC calculations based on school records and the fall 2012 baseline student survey. 

NOTES: The analyses reported in this table are based on the full study sample, which includes all 
students who agreed to participate in the study and were randomly assigned. Due to small numbers, 
percentages for the Native American demographic group are not included in the table. 

The estimated differences between the case-managed group and the non-case-managed group are 
regression adjusted using ordinary least squares, controlling for random assignment blocks by school. 
The values in the column labeled “Case-Managed Group” are the observed means for students randomly 
assigned to the case -managed group. The “Non-Case-Managed Group” values are the regression- 
adjusted means for students randomly assigned to the non-case-managed group, using the observed 
distribution of the case-managed group across random assignment blocks as the basis for the adjustment. 

Effect sizes are calculated by dividing the estimated difference by the standard deviation of the 
characteristic for students in the full study sample who are in the non-case-managed group. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between the case -managed and non-case-managed 
groups. Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

a These survey categories are not mutually exclusive. 

b Of those students who responded that English was not the predominant language at home, 92 
percent reported speaking Spanish at home. 

Respondents without siblings old enough for high school are omitted. 

d Scale based on responses to survey questions 13a-13e, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach's alpha = 0.83. 

e Scale based on responses to survey questions 6a-6g, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach's alpha = 0.81. 

f Scale based on responses to survey questions 12a-12f, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach's alpha = 0.89. 

g Scale based on responses to survey questions lOa-lOf, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach's alpha = 0.89. 

h Due to missing values, the number of students included varies by characteristic. The sample size 
reported here is for the full study sample. The average percentage of missing data on any given 
characteristic is 1 1 percent and ranges from 0 percent to 35 percent. The percentage of missing data is 
high on some characteristics because baseline school records data from the prior school year are not 
available for students in one of the districts in the study sample. 


data sources were high — over 90 percent in both cases. The differential response rates between 
the case-managed and non-case-managed students are small, with less than a 2 percentage point 
difference for each data source. Although the 1.8 percentage point difference in the survey 
response rate was statistically significant, it is small and is characterized as “low differential 
attrition” based on the current standards of the What Works Clearinghouse. These standards 
indicate that for overall attrition of 10 percent (which is slightly more attrition than for this 
sample), a difference between experimental groups of less than 6.3 percentage points qualifies 

3 

as “low attrition.” 


3 What Works Clearinghouse (2014). 
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Appendix Table B.2 

Study Sample Response Rates, by Follow-Up Data Source and Group 


Data Source for Outcomes (%) 

Case- 

Managed 

Group 

Non-Case- 

Managed 

Group 

Estimated 

Difference 

P- Value for 
Estimated 
Difference 

Spring 2013 follow-up student survey 

92.8 

94.6 

-1.8 * 

0.069 

School records data (2012-2013) 

90.3 

91.3 

-1.0 

0.209 

Sample size (N=2,230) 

1,140 

1,090 




SOURCE: MDRC calculations based on the spring 2013 follow-up student survey and student 
records obtained from school districts for the 2012-2013 school year. 


NOTES: The analyses reported in this table are based on the full study sample, which includes all 
students who agreed to participate in the study and who were randomly assigned. The estimated 
differences between the case-managed group and the non-case-managed group are regression 
adjusted using ordinary least squares, controlling for random assignment blocks by school. The 
values in the column labeled “Case-Managed Group” are the observed response rate for students 
randomly assigned to the case-managed group. The “Non-Case-Managed Group” values are the 
regression-adjusted response rates for students randomly assigned to the non-case-managed 
group, using the observed distribution of the case-managed group across random assignment 
blocks as the basis for the adjustment. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between case-managed and non-case-managed 
groups. Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 
percent. 
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About MDRC 


MDRC is a nonprofit, nonpartisan social and education policy research organization dedicated 
to learning what works to improve the well-being of low-income people. Through its research 
and the active communication of its findings, MDRC seeks to enhance the effectiveness of so- 
cial and education policies and programs. 

Founded in 1974 and located in New York City and Oakland, California, MDRC is best known 
for mounting rigorous, large-scale, real-world tests of new and existing policies and programs. 
Its projects are a mix of demonstrations (field tests of promising new program approaches) and 
evaluations of ongoing government and community initiatives. MDRC’s staff bring an unusual 
combination of research and organizational experience to their work, providing expertise on the 
latest in qualitative and quantitative methods and on program design, development, implementa- 
tion, and management. MDRC seeks to leam not just whether a program is effective but also 
how and why the program’s effects occur. In addition, it tries to place each project’s findings in 
the broader context of related research — in order to build knowledge about what works across 
the social and education policy fields. MDRC’s findings, lessons, and best practices are proac- 
tively shared with a broad audience in the policy and practitioner community as well as with the 
general public and the media. 

Over the years, MDRC has brought its unique approach to an ever-growing range of policy are- 
as and target populations. Once known primarily for evaluations of state welfare-to-work pro- 
grams, today MDRC is also studying public school reforms, employment programs for ex- 
offenders and people with disabilities, and programs to help low-income students succeed in 
college. MDRC’s projects are organized into five areas: 

• Promoting Family Well-Being and Children’s Development 

• Improving Public Education 

• Raising Academic Achievement and Persistence in College 

• Supporting Low-Wage Workers and Communities 

• Overcoming Barriers to Employment 

Working in almost every state, all of the nation’s largest cities, and Canada and the United 
Kingdom, MDRC conducts its projects in partnership with national, state, and local govern- 
ments, public school systems, community organizations, and numerous private philanthropies. 



